-- Logs begin at Wed 2022-07-27 06:04:00 UTC, end at Wed 2022-07-27 06:47:12 UTC. -- Jul 27 06:31:35.422241 dv-cinder-850989-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 06:31:35.430238 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 06:31:35 2022] *** Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: nodename: dv-cinder-850989-4-iscsi Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: machine: x86_64 Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: clock source: unix Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: pcre jit disabled Jul 27 06:31:35.432237 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: detected number of CPU cores: 4 Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: current working directory: / Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: detected binary path: /usr/bin/uwsgi-core Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: your processes number limit is 47680 Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: your memory page size is 4096 bytes Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: detected max file descriptor number: 2048 Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: lock engine: pthread robust mutexes Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: thunder lock: enabled Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 06:31:35.432677 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 06:31:35.474544 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: Python main interpreter initialized at 0x564ae543e9c0 Jul 27 06:31:35.474544 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: python threads support enabled Jul 27 06:31:35.474544 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: your server socket listen backlog is limited to 100 connections Jul 27 06:31:35.474544 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: your mercy for graceful operations on workers is 90 seconds Jul 27 06:31:35.474941 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 06:31:35.475011 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: *** Operational MODE: preforking *** Jul 27 06:31:35.475269 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: *** uWSGI is running in multiple interpreter mode *** Jul 27 06:31:35.475269 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: spawned uWSGI master process (pid: 86806) Jul 27 06:31:35.475472 dv-cinder-850989-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 06:31:35.476650 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: spawned uWSGI worker 1 (pid: 86819, cores: 1) Jul 27 06:31:35.476650 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: spawned uWSGI worker 2 (pid: 86820, cores: 1) Jul 27 06:31:35.476650 dv-cinder-850989-4-iscsi devstack@keystone.service[86806]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 06:31:40.977259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 06:31:40.977259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: from cryptography.utils import int_from_bytes Jul 27 06:31:40.978029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 06:31:40.978029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: from cryptography.utils import int_from_bytes Jul 27 06:31:40.982876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 06:31:40.982876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: from cryptography.utils import int_from_bytes Jul 27 06:31:40.983314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 06:31:40.983314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: from cryptography.utils import int_from_bytes Jul 27 06:31:41.027116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 06:31:41.027360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 06:31:41.027499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] command line args: [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 06:31:41.027770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 06:31:41.027948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ================================================================================ {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 06:31:41.028201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] admin_token = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.028443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] config_dir = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.028630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.028808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] config_source = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.028987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] debug = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.029189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.030080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.030080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.030080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.030274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.030446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 06:31:41.031269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_config_append = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 06:31:41.031444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_dir = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] command line args: [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 06:31:41.031576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_file = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 06:31:41.031698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ================================================================================ {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 06:31:41.031813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031874 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] admin_token = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.031939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] config_dir = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032126 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] config_source = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] debug = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.032873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033045 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033695 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033695 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.033973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.034119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.034248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.034248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_config_append = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.034511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.034652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_dir = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_file = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.035865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] publish_errors = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.036912 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.037986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] standard_threads = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038308 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038585 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038585 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.038971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] use_journal = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] use_json = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] use_stderr = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] use_syslog = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039906 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.039906 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.040156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.040358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.040490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] publish_errors = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.040616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.040743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.040879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.041013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.041140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.041273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.041401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.041527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.041665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.041800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.041934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.042151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] standard_threads = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.042289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.042416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.042554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.042683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.042683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.042929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.043058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] use_journal = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.043183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.043321 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] use_json = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.043512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.043650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] use_stderr = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.043650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] use_syslog = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.043898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 06:31:41.044172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.044944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.045851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.046884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.047935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048061 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048199 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048842 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.048977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.049918 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.050075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.050206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.050339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.051910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.052962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.053997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.054793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.055933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056468 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.056871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057268 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.057814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058770 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.058900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.059976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.060904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061030 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061160 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061816 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.061980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.062930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063590 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.063985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.064907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.065990 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.066912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067244 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067625 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.067890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068299 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068833 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.068971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.external = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069364 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.069910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.password = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] auth.token = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070636 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.070902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.071952 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.072947 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073603 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.073996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.external = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074277 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.074961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.password = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] auth.token = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.075883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.076933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.077863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.078869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079270 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.079952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.080889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.081018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.081147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.081147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.081390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.081536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082392 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082625 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082748 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.082957 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.083992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.084933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.085978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.086930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087587 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087657 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.087945 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.088938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089584 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089795 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.089956 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090311 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090468 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090537 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090680 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.090970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091621 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.091980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092344 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092634 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.092988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.093916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.094939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095298 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095671 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] role.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] role.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.095950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096183 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.096985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.097064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.097262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.097373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098660 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.098911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099315 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.099954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] role.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.100943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] role.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.101907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102388 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.102949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103197 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103667 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.103902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.104946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105199 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.105924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.106931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107163 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.107933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.connection = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.108934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.109857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110043 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.110988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111229 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.111929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.112936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.113026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.113331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 06:31:41.113413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.113525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.114060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.connection = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.114310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.114560 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.114800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.115043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.115282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.115523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.115763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.116003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.116250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.116733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.116969 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.117217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.117455 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.117701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.117933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.118194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.118432 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 06:31:41.118603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 06:31:41.142476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.142705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.142933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.143017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.143213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.143445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.151757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.152364 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.182003 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.182229 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.184078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.184317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.235465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.239676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.287016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.287248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.290009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.290248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.352520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.352657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.352735 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.352853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.352954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.353722 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.376890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.377876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.378025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.378111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.378243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.378326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.378477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.389290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.389607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.402395 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.402623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.402843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.402968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403206 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.403877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.404096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.404318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 06:31:41.420938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.421152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 06:31:41.421245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.421359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 06:31:41.421491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.421692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.421779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 06:31:41.421890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 06:31:41.422060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.422936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.433961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.434153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.440838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.441086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.445633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.445901 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.446009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446586 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.446853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.447914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.448101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.448184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.448368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.448449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.448650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.471596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.471736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.473607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.473796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.473897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.474104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.478494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.478769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.478853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.479046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.479129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.479329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.485613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.485851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.485980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.486270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.490454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.490748 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.490996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.491081 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491396 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.491939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492409 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.492956 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.508816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.509101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.509753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.510065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.513016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.514001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.516165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.516456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.517191 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.517499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.522390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.522589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.525792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.526744 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.529007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.529308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.529594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.529882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.529978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.530189 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.530286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.530481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.530566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.530805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.531082 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.531357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.546669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.546956 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.547224 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.547312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.547488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.547586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.547828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.548088 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.554516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.555298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.556075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.556354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.556827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.557103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.559226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.559513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.559815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.559927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.560101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.560210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.560393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.560507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.560794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.561071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.572397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.572682 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.572965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.573248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.573346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.573527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.573619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.573803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.573886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.574105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.574190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.574380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.574652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.574924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.590903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.591058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.594045 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.594977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.596978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.598902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.599469 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.600962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.601245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.601521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.603015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.604551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.604848 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.605157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.607984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.609564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.611701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.612483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.612769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.613440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.614288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.614565 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.616676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.616973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.617265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.617550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.622555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.622843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.623122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.625644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.625957 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.626241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.630492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.630771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.631048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.631321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.631589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.631865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.632135 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.632412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.632780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.633045 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.633329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.633598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.633859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.633955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 06:31:41.634160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.634246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.634445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.634725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.635011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.635286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.635562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.635841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.636217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.636501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.636790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.637064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.637331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.637604 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 06:31:41.668076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564ae543e9c0 pid: 86819 (default app) Jul 27 06:31:41.671451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebedab7-db69-4904-9de2-af15f9663b39 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:41.671743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebedab7-db69-4904-9de2-af15f9663b39 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:41.671986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564ae543e9c0 pid: 86820 (default app) Jul 27 06:31:41.672071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebedab7-db69-4904-9de2-af15f9663b39 None None] PATH_INFO: `/v3` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:41.676854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Jul 27 06:31:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:49.256453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:49.257036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:49.257036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:49.261451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:31:49.266614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:49.267033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:49.267180 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:49.337810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: INFO dbcounter [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] Registered counter for database keystone Jul 27 06:31:49.359390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG oslo_db.sqlalchemy.engines [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86819) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 06:31:49.366675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writer thread running {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 06:31:49.544609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG passlib.handlers.bcrypt [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86819) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 06:31:49.544943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG passlib.handlers.bcrypt [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86819) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 06:31:49.599114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:49.600751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:49.725487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 06:31:49.895237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:49.964283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: INFO dbcounter [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] Registered counter for database keystone Jul 27 06:31:49.983744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG oslo_db.sqlalchemy.engines [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86820) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 06:31:49.990858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writer thread running {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 06:31:50.107769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ee9b3-babf-469b-be5b-837bcc88bc9d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:50.111970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:50.113225 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:50.113763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:50.114068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:50.122063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:50.174060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING keystone.common.rbac_enforcer.enforcer [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 06:31:50.174354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1e95b-fc42-4891-a9c9-4f8671caa083 admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:50.184404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 06:31:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 06:31:50.191021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:50.205155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52123ef3-5d9e-4071-a468-305f57e9a243 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:50.209648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:50.210778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:50.211070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:50.211352 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] PATH_INFO: `/v3/projects/0d277a3b91684c12b74997019b6d7be2` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:50.212847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] RBAC: Authorizing `identity:get_project(project_id=0d277a3b91684c12b74997019b6d7be2)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:50.264356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING keystone.common.rbac_enforcer.enforcer [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 06:31:50.264703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e4777-c36b-4d78-9688-96a8744ba7b0 admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:50.269527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Jul 27 06:31:50 2022] GET /identity/v3/projects/0d277a3b91684c12b74997019b6d7be2 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:52.219470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed9fb-de4f-4751-8d16-28f780b8421a None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:52.220022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed9fb-de4f-4751-8d16-28f780b8421a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:52.220022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed9fb-de4f-4751-8d16-28f780b8421a None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:52.224662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:52.230071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:52.230494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:52.230634 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:52.252352 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:52.253886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:52.261536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 06:31:52.283274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ed9fb-de4f-4751-8d16-28f780b8421a None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:52.295662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ed9fb-de4f-4751-8d16-28f780b8421a None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:52.300406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:52.301011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:52.301296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:52.301575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:52.309147 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:52.310965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0313-f9a5-4333-89ef-cc8942da1fca admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:52.319761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jul 27 06:31:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 06:31:52.324989 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:52.334529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e78eb-da6d-45b7-af37-16f4a94de069 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:52.338895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:52.339529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:52.339816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:52.340098 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:52.341463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:52.343309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73560b9a-fb35-462f-97e4-af6f4be5ba49 admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:52.344870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Jul 27 06:31:52 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:54.386644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698bf-6c2c-4a9a-a344-2b983f1edb65 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.387177 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698bf-6c2c-4a9a-a344-2b983f1edb65 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.387265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698bf-6c2c-4a9a-a344-2b983f1edb65 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.388528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:54.394277 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.394681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.394842 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.402279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf64ff-2167-4b58-8752-e32db598ad65 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.402567 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf64ff-2167-4b58-8752-e32db598ad65 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.402841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf64ff-2167-4b58-8752-e32db598ad65 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.404115 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:31:54.409373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.409656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.409934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.413921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:54.415128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:54.450885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG passlib.handlers.bcrypt [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86820) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 06:31:54.451352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG passlib.handlers.bcrypt [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86820) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 06:31:54.493535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:31:54.497640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:54.499240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:54.517282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:54.540680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:54.598898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:31:54.621867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:54.625223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79cb62-7a42-432e-b2ad-e590a95de9ec None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:54.662127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:54.662726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.663015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.663693 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.674522 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:31:54.683410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:31:54.684312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:31:54.685155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:31:54.690647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:54.725624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c93d6-93fc-4267-bd12-6e33ed74f964 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:54.727932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:54.762334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:54.763210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.763500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.763784 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.770941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:54.789810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63890f-c3ce-4d8d-9469-2e8ddd7e22a3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:54.822018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:54.822638 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:54.822916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:54.823193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:54.823913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:31:54.824218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:54.825600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:54.831935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:54.833445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:54.837697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:31:54.843131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:54.933266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:31:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:31:54.938714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:54.941998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:54.943439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c3d-8d53-4739-bb79-48a86c8aa4e5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:55.045417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e40463-429e-427c-8d84-b2cab9931f79 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:55.079350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:55.080208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:55.081298 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:55.081803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:55.084073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:55.084708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:55.084989 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:55.085266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:55.086456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:55.087880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:55.089446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:31:54 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:31:55.094757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:55.100690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:55.102190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:55.109284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:31:55.120855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:31:55.121691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:31:55.122979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:31:55.122979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:31:55.122979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:31:55.122979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:31:55.122979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:31:55.128443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:55.183563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:55.194539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33f0e2-ecb8-4e09-8f24-d04f04bbd092 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:55.227049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:55.227686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:55.227812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207752ab-f5ac-49a0-b4dc-f7cebd655e0f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:55.227969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:55.228244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:55.229379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:55.230891 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:55.232438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:31:55.239251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:55.261192 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:55.261815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:55.262126 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:55.262403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:55.263098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:31:55.263399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:55.264776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628799-6ba7-4582-a579-15ea14b3696b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:55.281530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:31:55.321638 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:55.343467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48143c-e978-4457-b9f7-dbc8485d4e24 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:55.379358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:55.379994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:55.380280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:55.380560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:55.381366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.domain.id=default, user.id=cc98442be8894ee2a2dc3bab2659ad72) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:31:55.381690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:55.383188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddab7e-aed0-4e2f-a89b-b23a56979e1d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:55.390627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 06:31:55 2022] GET /identity/v3/role_assignments?user.id=cc98442be8894ee2a2dc3bab2659ad72&scope.domain.id=default&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:31:57.633542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adbe7-1415-412d-995a-9c87bb4d7b7e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:57.634623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adbe7-1415-412d-995a-9c87bb4d7b7e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:57.635125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adbe7-1415-412d-995a-9c87bb4d7b7e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:57.636851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:57.642201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:57.643532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:57.644006 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:57.678944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:57.680481 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:57.717883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c28f-545e-4c07-99be-160a211ab6db None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:57.718511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c28f-545e-4c07-99be-160a211ab6db None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:57.718999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c28f-545e-4c07-99be-160a211ab6db None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:57.720534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:31:57.726275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:57.726814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:57.727284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:57.762973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:57.764532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:57.787889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:31:57.822193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:57.888323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:31:57.922959 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:57.962361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.003052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9e392-4ab5-4588-ab72-991f2c4a3fcd None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.051120 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.060766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.061431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.061712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.062067 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.077159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:31:58.090690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:31:58.091609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:31:58.092362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:31:58.094546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 13/26] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:31:58.102880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.163656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe008625-2a75-4d63-806c-26018287ccaf None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.208606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.209250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.209537 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.209819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.211166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:58.212925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb576f7d-5d91-4536-9fd3-4f28c73b6ae1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:58.225508 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9eecb-97e5-4145-b91d-ba206b25f092 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.264828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.265885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.266474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.266941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.267962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:31:58.268492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:58.270183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c29ff-be3a-4b5a-b28d-eab3b1f3931d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:58.272370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.290340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 14/27] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:58.290340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788bdb-1e52-4d9c-b7ac-d35d933ff5d1 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.290340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788bdb-1e52-4d9c-b7ac-d35d933ff5d1 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.290340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f788bdb-1e52-4d9c-b7ac-d35d933ff5d1 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.290340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 15/28] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:58.290765 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389060-ae86-4963-b8dc-fc1a63cb3c6a None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.290765 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389060-ae86-4963-b8dc-fc1a63cb3c6a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.291051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389060-ae86-4963-b8dc-fc1a63cb3c6a None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.292321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 16/29] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:58.292917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389060-ae86-4963-b8dc-fc1a63cb3c6a None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.416108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389060-ae86-4963-b8dc-fc1a63cb3c6a None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.452983 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.454085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.454564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.455054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.457117 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:58.458871 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036899d-d3eb-4e14-8e28-5cd0cab24904 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:58.460709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 17/30] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:31:58 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:58.462080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.462603 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.463123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.467280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 14/31] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 06:31:57 2022] POST /identity/v3/roles => generated 219 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:31:58.468747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.469047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.469345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.485501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:58.487136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:58.493079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:58.494752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:58.556153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.567253 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 18/32] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:31:58.567943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.575340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 15/33] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:31:58.603765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.649529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.737763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f92f4-d832-4f54-bb7f-48f98b1a1537 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.779157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559849f-4e4f-4074-86f9-31bfdf40ceb0 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:58.792218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.793274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.793773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.794303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.795305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:58.797151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:58.821002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 16/34] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 06:31:58 2022] POST /identity/v3/roles => generated 225 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:31:58.821459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:58.822243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:58.822527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:58.822854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:58.823112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.827539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:58.827539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:58.827539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 19/35] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:31:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:58.836033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:58.937604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:58.994673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18263-a570-41f7-95f1-c8f6ef67c556 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.051974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:59.065644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce16c7-a9ef-416d-86b3-95c5b4b70a10 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.069304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.070363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.070874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.071408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] PATH_INFO: `/v3/domains/default/users/cc98442be8894ee2a2dc3bab2659ad72/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.074834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc98442be8894ee2a2dc3bab2659ad72, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.077021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995f32b-d5ff-4bcd-a54f-bfd9911d657e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.107272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.107928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.108221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.108497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.120609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.122137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:31:59.140772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:31:59.141666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:31:59.143810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:31:59.143810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:31:59.143810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:31:59.143810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:31:59.143810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 17/36] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:31:59.150078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:59.231616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 20/37] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Jul 27 06:31:58 2022] PUT /identity/v3/domains/default/users/cc98442be8894ee2a2dc3bab2659ad72/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:31:59.233681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2947d-9046-481c-84b9-951605e73543 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.234091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2947d-9046-481c-84b9-951605e73543 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.234452 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2947d-9046-481c-84b9-951605e73543 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.239320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 21/38] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:59.245300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee28b-6cff-4041-879e-40c338b702a7 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.245520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee28b-6cff-4041-879e-40c338b702a7 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.245981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee28b-6cff-4041-879e-40c338b702a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.265789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-16eee28b-6cff-4041-879e-40c338b702a7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:59.266768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23790c-f8f4-4c49-9fe5-43ae0b986988 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.268162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-16eee28b-6cff-4041-879e-40c338b702a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:59.314984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.315924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.317184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.317700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.318664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:31:59.319736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.322192 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6e10-a4af-46bc-8cfa-ced65cbb18cb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.336862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 18/39] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:59.343263 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f5f-3be6-4dea-b5c4-815c1529ca27 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.343648 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f5f-3be6-4dea-b5c4-815c1529ca27 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.343993 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f5f-3be6-4dea-b5c4-815c1529ca27 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.345435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 19/40] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:31:59.350139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60f5f-3be6-4dea-b5c4-815c1529ca27 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:59.350292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 22/41] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:31:59.352213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.352650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.353110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.364786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:59.375913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:31:59.377444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:31:59.435885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:59.459527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 23/42] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:31:59.460389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:59.490306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60f5f-3be6-4dea-b5c4-815c1529ca27 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.531861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.532411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.532699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.532979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.534588 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.536068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.568455 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 20/43] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:31:59 2022] POST /identity/v3/projects => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:31:59.569324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:59.569616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e852c3-debd-41c3-96d1-a4b89bef28d0 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.626806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.628486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.629210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.630499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.632718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.635812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.639209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 24/44] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:31:59.646208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:31:59.658861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:59.711678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792704c-31c2-4218-a9bf-afced1c3405b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.726016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:31:59.759572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.760322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.760676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.760966 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.761699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.763261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34954098-cd27-4af6-b2cc-2ddc52792022 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.781718 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 21/45] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 06:31:59 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:31:59.793327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66aeb4-5fba-4488-9ba0-5c810a0ae13d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:31:59.835216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:31:59.836267 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:31:59.836790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:31:59.837334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:31:59.838332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:31:59.840206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe93b6-99e7-48c0-a9ff-055cc9e7ee7a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:31:59.874571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 25/46] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:31:59 2022] POST /identity/v3/projects => generated 312 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:01.259292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd4666-0a75-4cea-8a40-26bcad839e2a None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.259840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd4666-0a75-4cea-8a40-26bcad839e2a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.259840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd4666-0a75-4cea-8a40-26bcad839e2a None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.261188 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 22/47] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:01.266301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565341-34f0-40a8-9356-67213563b56a None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.266733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565341-34f0-40a8-9356-67213563b56a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.266878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4565341-34f0-40a8-9356-67213563b56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.285023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d4565341-34f0-40a8-9356-67213563b56a None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:01.286247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d4565341-34f0-40a8-9356-67213563b56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:01.351291 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 26/48] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:01.375404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd4666-0a75-4cea-8a40-26bcad839e2a None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:01.482129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd4666-0a75-4cea-8a40-26bcad839e2a None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:01.502854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:01.517327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:01.517967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.518254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.518532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.529479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:01.535986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:01.536996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:01.537867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 23/49] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:01.542958 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4565341-34f0-40a8-9356-67213563b56a None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:01.574787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:01.642736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4565341-34f0-40a8-9356-67213563b56a None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:01.678670 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:01.679303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.679588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.679869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.680596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:01.680914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:01.682419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:01.691170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 27/50] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:01.696654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:01.795520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfb60f-6418-4426-a4f3-053297fb836d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:01.827737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:01.828371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.828653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.828930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.830385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:01.831828 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c7ecd-0acd-4292-a4f8-83dfabbb5881 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:01.833442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 24/51] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:01 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:01.838995 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:01.917464 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526e26-a02d-4c27-8d79-04d0fdb48b3a None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.917761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526e26-a02d-4c27-8d79-04d0fdb48b3a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.918072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526e26-a02d-4c27-8d79-04d0fdb48b3a None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.919422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 25/52] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:01.924961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.925248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.925534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.937841 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:01.942960 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:01.944161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:01.945318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755db8a-5777-4c82-894f-993ceadb469f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:01.979232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:01.979877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:01.980167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:01.980450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:01.981598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:01.983065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:01.984665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 28/53] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:01.988570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:01.991480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:02.011907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 26/54] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:02.035285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:02.087802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c1adf-f1c3-494e-b31d-9ae893f4cebb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:02.120930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:02.121555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:02.121844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:02.122156 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:02.122950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.domain.id=default, user.id=cc98442be8894ee2a2dc3bab2659ad72) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:02.123270 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:02.124702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:02.130677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 29/55] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jul 27 06:32:01 2022] GET /identity/v3/role_assignments?user.id=cc98442be8894ee2a2dc3bab2659ad72&scope.domain.id=default&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:02.131549 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd062-c74a-496a-a921-dd6147937334 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:02.168445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:02.169106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:02.169390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:02.169669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:02.170895 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:02.172367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88498d3d-9064-45e6-90b3-968ed1d7b472 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:02.175843 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 27/56] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:02.181965 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:02.211346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:02.288263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f63-198e-4282-ad60-eaf915e67646 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:02.322150 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:02.322779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:02.323066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:02.323346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:02.324041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:02.325480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154b77-c79e-4c6c-9969-2c162de2941e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:02.352357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 30/57] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 06:32:02 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:04.334033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde8e-eb5d-4e83-8513-710997536f61 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:04.334563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde8e-eb5d-4e83-8513-710997536f61 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:04.334666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde8e-eb5d-4e83-8513-710997536f61 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:04.335940 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 28/58] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:04.341100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:04.341510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:04.341674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:04.360693 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:04.361979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:04.427412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 31/59] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:04.450990 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde8e-eb5d-4e83-8513-710997536f61 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:04.499296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:04.552680 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde8e-eb5d-4e83-8513-710997536f61 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:04.586179 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:04.586746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:04.587034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:04.587317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:04.588467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:04.589913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a022c11-cf1a-4ec3-88e8-29a1abd1b88e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:04.591497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 29/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:04.597612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:04.607381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:04.696107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445d23-4e90-423e-9614-fc5b687a6466 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:04.729515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:04.730160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:04.730447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:04.730723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:04.731406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:04.732839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d8318-cb16-40a3-8e3e-379e98842487 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:04.758606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 32/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:04 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:07.926347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2a19c-d763-461f-918e-5154f149e7e6 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:07.926963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2a19c-d763-461f-918e-5154f149e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:07.926963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2a19c-d763-461f-918e-5154f149e7e6 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:07.928422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 30/62] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:07.934434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:07.934907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:07.934981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:07.953779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:07.955392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:08.032187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.045120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 33/63] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:08.077744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c7ae-47ed-4c82-8493-b3707c0d60b2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.078096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c7ae-47ed-4c82-8493-b3707c0d60b2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.078381 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c7ae-47ed-4c82-8493-b3707c0d60b2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.080733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 31/64] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:08.081771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.089135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.089426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.089691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.115078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:08.116302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:08.222011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 32/65] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:08.251918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.302336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94121941-f4e3-4257-8d7e-9e213eee5837 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.372417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.373074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.373360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.374089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.397498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.401531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6f2ff500-ec77-46a2-982c-e45c3c7cb24e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:08.416709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:08.417632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:08.418518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 34/66] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:08.419739 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcea5f-5c95-4e3d-b632-88171c007c5f None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.420027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcea5f-5c95-4e3d-b632-88171c007c5f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.420318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcea5f-5c95-4e3d-b632-88171c007c5f None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.428536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 35/67] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:08.429497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcea5f-5c95-4e3d-b632-88171c007c5f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.477684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06f3b9-3e04-45e5-be87-41046c7e6a99 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.516598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.517491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.517848 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.518351 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.524728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.530795 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f9c10a3c-b013-44e2-9731-08b1594edcad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:08.536677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:08.537536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:08.538343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 33/68] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:08.539804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838917e-8296-4aaa-bdd7-d66daaa9bdf1 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.539804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838917e-8296-4aaa-bdd7-d66daaa9bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.539804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838917e-8296-4aaa-bdd7-d66daaa9bdf1 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.541209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 34/69] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:08.542421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d0f8-c86e-43c6-8743-f918bbb38e0f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.543055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d0f8-c86e-43c6-8743-f918bbb38e0f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.543339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d0f8-c86e-43c6-8743-f918bbb38e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.569535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6c39d0f8-c86e-43c6-8743-f918bbb38e0f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:08.571463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6c39d0f8-c86e-43c6-8743-f918bbb38e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:08.582231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcea5f-5c95-4e3d-b632-88171c007c5f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.622258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.622970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.623262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.623545 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.624416 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:08.624992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:08.626669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:08.643750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 36/70] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:08.643750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.663454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 35/71] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:08.664788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.665061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.667756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.694643 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:08.695880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:08.722438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.757233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5502a-e6c8-4cfc-bbe3-db04c77e3c74 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.763388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 36/72] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:08.764407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.784285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.802787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.803419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.803709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.803997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.804720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:08.805036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:08.806463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f521a96-c501-47a3-9e6b-7d21a9bf0eb9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:08.814422 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 37/73] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:08.819250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880a5a-f068-4e8f-a4cd-38130c17345c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.819539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880a5a-f068-4e8f-a4cd-38130c17345c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.819818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880a5a-f068-4e8f-a4cd-38130c17345c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.821112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 38/74] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:08.821112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21880a5a-f068-4e8f-a4cd-38130c17345c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.871570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59740e52-39a5-4f0e-8469-1200c9d27cca None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.905465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.906148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.906435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.906721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.908655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:08.910151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:08.911751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 37/75] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:08.911751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:08.924061 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21880a5a-f068-4e8f-a4cd-38130c17345c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:08.944284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:08.957846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:08.958522 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:08.958809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:08.959091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:08.969118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:08.975747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:08.976583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:08.977404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 39/76] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:08.978076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.016410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.021311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b57bfe-71c1-4ab1-a8bf-1edc85e79e8d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.055799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.056439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.056730 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.057013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.058190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.059629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a23252-7c24-4abd-bb0c-d26a270a66a2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.061504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 38/77] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:09.062663 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.062952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.063233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.080853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84055cd9-3549-46f3-b295-a43e5d57b534 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.081010 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:09.082223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:09.114173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.114817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.115101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.115379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.128622 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.130122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.131760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 40/78] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:08 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:09.132521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.151133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 39/79] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:09.152098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.181386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.232256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.238750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f42f-1eba-478e-affa-04127543e2f4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.249658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd3e4-df3f-4619-9444-7ba5f00c8356 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.272435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.273011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.273293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.273566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.283847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.284461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.284743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.285017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.285728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:09.286073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.287439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.298565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 40/80] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:09.303828 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.340512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.341985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.394662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.403825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a5629-0d6b-4d0b-bd38-fbd7aec4b6fb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.431910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.437572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.438220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.438509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.439301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.439986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.440312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 41/81] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:09.441148 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.441425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.469033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 41/82] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:09 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:09.475030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.552154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44216650-597e-4c7e-b48b-97ab99982c7f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.590506 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.590715 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15260d-a7ec-45e1-bf5b-b85265a3f475 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.591152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.591467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.591751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.630774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.631433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.631725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.631993 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.646194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:09.651858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:09.652746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:09.653516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 42/83] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:09.654385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.672330 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.672903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.683337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.684682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.791209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943ec25-7ce7-43ce-ae14-1b16ce475cb8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.815916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 42/84] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:09.816702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.830021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.830684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.830974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.831251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.832490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.833978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.835763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 43/85] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:09.836443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:09.929254 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.936262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:09.941364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994856fc-6372-4ea8-b916-f5f88276f677 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.962759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2a417-a881-4d2e-b7c5-f3c32a47fb29 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:09.979818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:09.980589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:09.981052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:09.981396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:09.982932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:09.984453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:09.987117 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 43/86] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:09.988223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.001323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.002079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.002373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.002658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.003391 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:10.003706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.005201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.014870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 44/87] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.015917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.096013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9f08-cbd7-4ef5-b3e0-71c010d8eeb9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.125886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23c136-c830-42bf-9ffa-b4d868aec4cf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.133233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.134110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.134478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.134821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.136155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.137761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.139950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 44/88] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:09 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.140941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.162284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.162920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.163273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.163556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.174372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:10.179909 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:10.250426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.253069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.275364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd438e-ee56-426a-9c45-6e5e27a6d57c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.320567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.321307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.321887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.322205 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.323282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=cc98442be8894ee2a2dc3bab2659ad72) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:10.324026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.326008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.334370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 45/89] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/role_assignments?user.id=cc98442be8894ee2a2dc3bab2659ad72&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.335369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.423883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 45/90] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:10.426229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.517996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:10.531526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:10.544261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117cd20-f852-4071-b948-2d218770a4c1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.599323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1466a9-9a0a-4531-a2f1-8e87ca701fea None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.636295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.637245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.637731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.638355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.639336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:10.639831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.641472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.650952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 46/91] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.652084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.652191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.652977 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.653277 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.653550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.654784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.656252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.659189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 46/92] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.671966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.780916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35eb42-c5b9-417c-a916-ff879d5c99c3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.790632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:10.847770 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.848750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.849222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.849722 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:10.851292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:10.852959 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:10.859955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 47/93] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:10.866527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:10.921585 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f4cf4-39dc-4817-94e5-ac4a500817e6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:10.962515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:10.963468 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:10.963941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:10.964429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.004361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:11.049811 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:11.051721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:11.103891 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259d11a-7754-44b7-98a0-416209d93e26 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:11.160770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 47/94] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:11.170497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:11.197515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:11.198249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:11.198562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:11.199000 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.199628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:11.199943 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:11.201388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:11.217045 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 48/95] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:10 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:11.254394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:11.290247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dedde-ac90-4bfe-ade7-e5c228997eef None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:11.336572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:11.337517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:11.338002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:11.338463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] PATH_INFO: `/v3/projects/3a0d259190614f00af9efe9709ae53f6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.342666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] RBAC: Authorizing `identity:get_project(project_id=3a0d259190614f00af9efe9709ae53f6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:11.344321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123f90-f636-4d5b-9173-e99c922f2238 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:11.346872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 48/96] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:11 2022] GET /identity/v3/projects/3a0d259190614f00af9efe9709ae53f6 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:11.352870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:11.409848 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:11.484612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612059-03a3-4438-b34a-e3fa7899b8ef None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:11.523420 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:11.524068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:11.524354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:11.524637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.525443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=3a0d259190614f00af9efe9709ae53f6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:11.525760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:11.527252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209b646-9647-43ba-bac1-3124d6ffb959 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:11.534767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 49/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:11 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=3a0d259190614f00af9efe9709ae53f6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:11.811244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fede85-0aea-46b9-b5ed-e9da82835785 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:11.811879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fede85-0aea-46b9-b5ed-e9da82835785 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:11.812366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fede85-0aea-46b9-b5ed-e9da82835785 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.814694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 49/98] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:11.820235 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:11.820529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:11.820820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:11.851016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:11.852240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:11.939965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 50/99] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:11.974956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fede85-0aea-46b9-b5ed-e9da82835785 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.111187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fede85-0aea-46b9-b5ed-e9da82835785 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:12.147132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:12.154395 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:12.155046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.155332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.155615 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.174816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-bf585fb3-58c4-4b50-bbfb-6e799b483e6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:12.186417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:12.187289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:12.189013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 50/100] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:12.194349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.232530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:12.341920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c7975-e9c1-4793-b053-2207094ebaf5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:12.385641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:12.386256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.386532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.386813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.387539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:12.387847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:12.388592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705bd9-b224-44f4-932e-afc669ab13b4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.389099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705bd9-b224-44f4-932e-afc669ab13b4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.389267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:12.389724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705bd9-b224-44f4-932e-afc669ab13b4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.391953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 51/101] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:12.397251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.397891 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.398431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.401378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 51/102] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:12.409195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.431820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:12.433013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:12.551610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 52/103] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:12.562284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:12.570381 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9ea5c-ecc7-4b64-93ae-790822de2684 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:12.583919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.636431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:12.637087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.637372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.637651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] PATH_INFO: `/v3/users/48a55fdc97024ca18b3768649cfd8327` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.676568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] RBAC: Authorizing `identity:get_user(user_id=48a55fdc97024ca18b3768649cfd8327)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:12.678055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:12.682831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 52/104] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:12 2022] GET /identity/v3/users/48a55fdc97024ca18b3768649cfd8327 => generated 289 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:12.690942 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.774397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a61568-ba5f-446b-b594-1652fc59c4ca None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:12.815699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:12.841010 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:12.841675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.841990 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.842279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.844092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:12.846460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:12.848482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 53/105] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:12.854240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:12.860889 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a511ad3-7f16-4e46-ab33-9ed6ced555dd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:12.912452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:12.913505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:12.914002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:12.914474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:12.948815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:13.046678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:13.048807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:13.065646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4a1d0-3ad6-4369-8e36-94316b67fd99 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:13.147260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:13.148692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.149125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.149662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.150544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:13.152256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9e6ae-2b4c-4f83-b85f-2449f157c35c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:13.213524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 53/106] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:13.216493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 54/107] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:32:12 2022] POST /identity/v3/projects => generated 302 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:32:13.231519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:13.295698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:13.300943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da779-6769-432b-b930-78c115c7335f None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.301240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da779-6769-432b-b930-78c115c7335f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.301505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2da779-6769-432b-b930-78c115c7335f None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.303063 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 55/108] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:13.310700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.311040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.311275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.368025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:13.370053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:13.393762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae65312-384e-406a-8d59-ee267ecdcd88 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:13.458461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:13.464137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:13.464806 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.465094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.465380 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.466961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:13.468460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:13.470800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 54/109] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:13 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:13.479335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:13.498289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 56/110] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:13.523461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:13.608158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa09b3-f730-4a5d-b346-bf893fe0671f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:13.627419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:13.660869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:13.660869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.660869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.660869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.661801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=48a55fdc97024ca18b3768649cfd8327) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:13.662762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:13.670246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:13.677011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 55/111] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:13 2022] GET /identity/v3/role_assignments?user.id=48a55fdc97024ca18b3768649cfd8327&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:13.692637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b94d34-5c4a-4758-9cc9-a685e2e5567f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:13.739892 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:13.740477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.740757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.741031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.756880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:13.762071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:13.763005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:13.763762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 57/112] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:13.769410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:13.910348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c7f85-4c88-403b-848e-58593adcdf72 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:13.950460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:13.951139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:13.951426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:13.951707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:13.952478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:13.952792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:13.954252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189e8c7-3353-4c13-8be9-f7cb3e6ba19c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:13.963076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 56/113] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:13.968821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.014483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.040709 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216011ff-cf6d-4339-bcf7-626bce86ee52 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.041222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216011ff-cf6d-4339-bcf7-626bce86ee52 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.041933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216011ff-cf6d-4339-bcf7-626bce86ee52 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.043479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 57/114] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:14.048507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6e0b-a927-4795-bb35-321851ed63ba None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.048792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6e0b-a927-4795-bb35-321851ed63ba None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.049073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6e0b-a927-4795-bb35-321851ed63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.083975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b80f6e0b-a927-4795-bb35-321851ed63ba None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:14.085228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b80f6e0b-a927-4795-bb35-321851ed63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:14.130550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2fb6-813b-4543-a9e6-5dc651690c97 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.166324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.190279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 58/115] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:14.191754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bd87-5a24-44ce-8411-6c58b2cb985f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.192048 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bd87-5a24-44ce-8411-6c58b2cb985f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.192377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bd87-5a24-44ce-8411-6c58b2cb985f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.193686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 59/116] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:14.199276 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.199565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.199846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.218392 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:14.219781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:14.242678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.243351 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.243641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.243927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.246712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:14.248173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:14.249978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 58/117] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:13 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:14.250833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.374894 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf41e-d81f-43b9-be15-33475bea07b8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.381233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 60/118] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:14.382412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.392276 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.416079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.417097 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.417473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.417831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.430147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:14.435503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:14.436358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:14.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 59/119] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:14.437782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.544597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d51e9f-440c-47ef-b0fa-dbf4608ea6d4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.585417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.586287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.586675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.587100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.599769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:14.605143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:14.606100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:14.607332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 60/120] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:14.608243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.650097 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.671904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba3b1d-e899-422c-9999-cd99244ce9ef None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.673777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.734360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.735031 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.735395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.735671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.737974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:14.739411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:14.742752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 61/121] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:14.743534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.770372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d2de-ff74-48f3-bd47-61b9300dc0f4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.830659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.831288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.831571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.831849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.832572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:14.832884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:14.834280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:14.856096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 61/122] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:14.856669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.924654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97cd6-aa08-49d8-aadd-dac5e5e0109c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:14.966757 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:14.967387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:14.967672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:14.967952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:14.968681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:14.968996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:14.969624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:14.971552 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:14.983020 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 62/123] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:14.984058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:14.984184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1dbe-62d5-4bfa-82bf-f88474f4b1b9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.023628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.030776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.031467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.031822 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.032209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/cc98442be8894ee2a2dc3bab2659ad72/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.035289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, user_id=cc98442be8894ee2a2dc3bab2659ad72, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.037021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf93cb0-c377-42d6-b654-a2e49864b9bb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.056977 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 62/124] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:14 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/cc98442be8894ee2a2dc3bab2659ad72/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:15.058319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc21c1-75ff-4e82-9682-f1e59792acaf None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.058617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc21c1-75ff-4e82-9682-f1e59792acaf None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.058897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc21c1-75ff-4e82-9682-f1e59792acaf None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.060173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 63/125] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:15.060893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc21c1-75ff-4e82-9682-f1e59792acaf None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.176301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f50d8-e7cc-4c5a-9c47-084d009dbdae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.189296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc21c1-75ff-4e82-9682-f1e59792acaf None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.243761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.244507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.244698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.244976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.246871 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.248359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426e61a-9d78-4b4c-835b-df441a4a5092 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.252459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 64/126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:15 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:15.253726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.254171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.254560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.269146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.269834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.270136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.270416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.272669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.274205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.275937 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 63/127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:14 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:15.276734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.293123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:15.294407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:15.299168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.388020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 65/128] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:15.389051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.394470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.398389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc022-e2bf-4867-9238-d627e3d8e9af None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.457202 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.457767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.458081 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.458357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.459527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.460964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.462931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 64/129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:15 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:15.463513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.549347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cb885-9590-423a-ba01-898126b545e1 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.613237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295daafa-97f3-4dc0-8a24-ef04abe06954 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.627626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.628301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.628582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.628856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.630157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.631608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.638556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 66/130] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:15 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:15.639428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.649018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.649672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.649975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.650261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.661578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:15.667507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:15.670102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 65/131] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:15.673156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.678804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.702088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.777870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e45526-fcf3-4f12-9b06-35ef4bebb638 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.813782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.814750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.815153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.815534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/64192fa2adce46b7a1ac80ee290def63/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.817787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, user_id=64192fa2adce46b7a1ac80ee290def63, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.819302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:15.838814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 66/132] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:15 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/64192fa2adce46b7a1ac80ee290def63/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:15.840983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:15.902869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a602c-3bde-4657-bff0-792fbf78680b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.970528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:15.982263 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b201f-10f3-4189-892d-55f4e2bf4a1f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:15.982873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:15.983804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:15.984288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:15.984766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/64192fa2adce46b7a1ac80ee290def63/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:15.987778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, user_id=64192fa2adce46b7a1ac80ee290def63, role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:15.989510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b315e-5ad9-4821-8701-49895f2f792d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.015820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 67/133] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:15 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/64192fa2adce46b7a1ac80ee290def63/roles/150d91154f9544c7a914d19f74d59fe8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:16.024193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2667-5293-4b33-9d2a-78fc467b5330 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.027214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2667-5293-4b33-9d2a-78fc467b5330 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.027737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2667-5293-4b33-9d2a-78fc467b5330 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.029783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 68/134] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:16.038275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.038931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.039418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.084986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.086098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.087749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.088246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.089222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:16.090258 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:16.092445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:16.094258 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.095461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:16.120051 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 67/135] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:16.131000 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.180066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:16.186560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 69/136] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:16.217520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.374445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112cc3a-9674-4273-9250-e00e393c79a8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:16.428894 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae1e7-f662-4e2e-a8b3-1434e2a2fc80 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:16.441705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.442662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.442947 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.443228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.444563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:16.446764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b3e9-b936-4268-864b-f20d6362b95f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.448831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 68/137] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:16 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:16.450026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e10ef-a643-4307-8900-cdd876b2c01e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.450323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e10ef-a643-4307-8900-cdd876b2c01e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.450607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e10ef-a643-4307-8900-cdd876b2c01e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.452259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 69/138] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:16.457906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e10ef-a643-4307-8900-cdd876b2c01e None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.478088 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.479031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.479358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.479600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.481389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:16.482007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:16.483491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f34a2c-151d-4714-9b49-25f507812fe0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.486359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 70/139] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:16.487759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.488046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.488332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.508381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:16.509574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:16.591288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e10ef-a643-4307-8900-cdd876b2c01e None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:16.591773 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:16.602641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 71/140] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:16.603527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.647163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.648156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.648735 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.649673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] PATH_INFO: `/v3/projects/3a0d259190614f00af9efe9709ae53f6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.651121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] RBAC: Authorizing `identity:get_project(project_id=3a0d259190614f00af9efe9709ae53f6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:16.655090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.657163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 70/141] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:16 2022] GET /identity/v3/projects/3a0d259190614f00af9efe9709ae53f6 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:16.663292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.747676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a56c-cea4-489b-9474-e3b8ae5b11df None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:16.814084 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.815843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.816487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.818112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.819018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:16.820529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:16.821859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:16.848091 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f9fe7-55f9-495a-97b9-ef8c5eaa674b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:16.863004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 72/142] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:16 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:16.863818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.924311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:16.924899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:16.925181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:16.925459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:16.952743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:16.959323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:16.960241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:16.961023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 71/143] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:16.969726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:16.990540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:17.115034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79f76-dfba-4db7-81b5-a5a4fada3923 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:17.183401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:17.184021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.184302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.184575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] PATH_INFO: `/v3/projects/3a0d259190614f00af9efe9709ae53f6/users/64192fa2adce46b7a1ac80ee290def63/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.187850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0d259190614f00af9efe9709ae53f6, user_id=64192fa2adce46b7a1ac80ee290def63, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:17.189368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:17.211337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9679a1b-99f0-4109-94c9-3fc12bbe0004 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:17.225296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 73/144] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:16 2022] PUT /identity/v3/projects/3a0d259190614f00af9efe9709ae53f6/users/64192fa2adce46b7a1ac80ee290def63/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:17.268267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:17.269332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.269917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.270429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.271405 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:17.272054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:17.273726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:17.288944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 72/145] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:17.295119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:17.460513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1143c-b445-4544-a3f0-cd95941ab886 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:17.480339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:17.509793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:17.510482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.510768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.511049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] PATH_INFO: `/v3/users/48a55fdc97024ca18b3768649cfd8327` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.513027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=48a55fdc97024ca18b3768649cfd8327)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:17.514553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:17.516231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 74/146] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:17 2022] GET /identity/v3/users/48a55fdc97024ca18b3768649cfd8327 => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:17.522917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:17.534835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:17.632771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c1d19-4f76-4105-9516-1dc5169a134a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:17.670255 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:17.670918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.671215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.671497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.672707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:17.674194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c4881-c2bb-44ba-b28d-3bd217eddc64 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:17.675884 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 73/147] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:17 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:17.681720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:17.793208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389fec1-083b-4a63-b9b4-ec183699d3d1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:17.830613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:17.831356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.831645 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.831978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/48a55fdc97024ca18b3768649cfd8327/roles/9838df12a6f345f0a80ce47deaca2479` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.834005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, user_id=48a55fdc97024ca18b3768649cfd8327, role_id=9838df12a6f345f0a80ce47deaca2479)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:17.835460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2d660-ecb8-44dc-bb04-85c69c4c0817 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:17.853039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 75/148] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:17 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/users/48a55fdc97024ca18b3768649cfd8327/roles/9838df12a6f345f0a80ce47deaca2479 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:17.879363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d85-46ae-4bb7-b8fd-1edc9cc3a836 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.879674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d85-46ae-4bb7-b8fd-1edc9cc3a836 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.879939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d85-46ae-4bb7-b8fd-1edc9cc3a836 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.881454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 74/149] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:17.887109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:17.887420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:17.887669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:17.907583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:17.908797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:17.951759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:17.997817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 76/150] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:18.039453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab3d85-46ae-4bb7-b8fd-1edc9cc3a836 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:18.131991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:18.162467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab3d85-46ae-4bb7-b8fd-1edc9cc3a836 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:18.204839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:18.205512 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.205798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.206139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.224408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:18.229476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:18.230514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:18.231463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 75/151] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:18.239142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:18.390910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a410b-13c4-405b-a262-f31ac8199ea7 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:18.435399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:18.436319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.436738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.437151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.438631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:18.439153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:18.440675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:18.453796 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 77/152] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:18.459732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:18.598827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c6383-26b2-4c4d-8d50-d5ba3eb66fc7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:18.640540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:18.641929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.642163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.642545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] PATH_INFO: `/v3/users/cc98442be8894ee2a2dc3bab2659ad72` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.646659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cc98442be8894ee2a2dc3bab2659ad72)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:18.648136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29a370-619a-4a22-9a04-3ef303ab4dc2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:18.649798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 76/153] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:18 2022] GET /identity/v3/users/cc98442be8894ee2a2dc3bab2659ad72 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:18.655595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:18.706049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:18.793496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a7392-4cca-453b-bde1-fa05ee449d2c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:18.834214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:18.834839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.835166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.835446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.901977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c931be-b634-4bbb-bc57-fb5e0368eab2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.902550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c931be-b634-4bbb-bc57-fb5e0368eab2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.903009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c931be-b634-4bbb-bc57-fb5e0368eab2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.904547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 77/154] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:18.915235 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:18.915611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:18.915964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:18.926896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:18.928383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:18.931904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:18.963798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:18.965011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:19.019824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:19.080739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 78/155] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:19.137834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:19.158307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 78/156] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:19.167015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:19.339724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ffe46-9b1c-4eeb-8ac8-399f66e541f4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:19.378300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:19.380708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2ce0-806a-4cb9-85a3-371666da9853 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:19.381503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df938db-58ac-4857-9034-de728479a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:19.382560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df938db-58ac-4857-9034-de728479a058 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.383083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df938db-58ac-4857-9034-de728479a058 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.383559 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df938db-58ac-4857-9034-de728479a058 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.384968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df938db-58ac-4857-9034-de728479a058 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:19.386640 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df938db-58ac-4857-9034-de728479a058 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:19.389382 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 79/157] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:19 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:19.399109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df938db-58ac-4857-9034-de728479a058 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:19.487871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:19.491144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.491439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.491705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.493934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:19.514904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:19.520946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:19.521919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:19.522720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 79/158] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:19.534919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:19.593803 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df938db-58ac-4857-9034-de728479a058 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:19.687232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:19.687900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.688216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.688496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.689302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=cc98442be8894ee2a2dc3bab2659ad72) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:19.689618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:19.691095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:19.708483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 80/159] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:19 2022] GET /identity/v3/role_assignments?user.id=cc98442be8894ee2a2dc3bab2659ad72&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:19.782319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27ab1-dd34-4d84-8c8f-d7895cf609ca None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:19.840226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:19.840865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.841140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.841410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.842163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:19.842472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:19.843863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02dd8-b962-43c8-b128-2087532c0f88 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:19.858651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 80/160] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:19.864310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:19.879955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde76d-a385-4965-9f80-f2864fdc6240 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.880266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde76d-a385-4965-9f80-f2864fdc6240 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.880506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde76d-a385-4965-9f80-f2864fdc6240 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.881875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 81/161] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:19.889426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:19.889714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:19.890014 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:19.919543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:19.920769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:20.019758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:20.047150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:20.081096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 82/162] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:20.105630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a679a-339c-4e5e-ad61-674d0a594db1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:20.135455 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:20.163583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:20.164237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:20.164519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:20.164792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:20.170689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:20.172145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:20.175020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 81/163] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:19 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:20.180233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:20.339974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d27bd-07c4-477f-9dda-4dd586d017e7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:20.340801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047e97a-3950-44a7-8631-1ddb9e842a21 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:20.387856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:20.395052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:20.395812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:20.396090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:20.396604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:20.418631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:20.419291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:20.419576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:20.419849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:20.444941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:20.449991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:20.450856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:20.451642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 83/164] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:20.458953 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:20.539031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:20.540525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:20.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:20.680056 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fc2c8-147d-499b-93df-8e82b9a6cd72 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:20.771994 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:20.772629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:20.772914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:20.773193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:20.773925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:20.774255 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:20.775634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:20.788719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 82/165] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:20.793159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 84/166] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:20.798447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:20.800155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:20.859221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:20.949297 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a98822-8b07-419c-ac8c-be7154770b2b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:20.973043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:21.042766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.043446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.043732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.044009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.046497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:21.048600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7166c35-c6e5-4817-8555-bb22836d8415 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:21.050384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 85/167] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:20 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:21.053093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d2eb-8655-4456-9cb5-3b7f5ec0540f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.053382 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d2eb-8655-4456-9cb5-3b7f5ec0540f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.053658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0d2eb-8655-4456-9cb5-3b7f5ec0540f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.055059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 86/168] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:21.061918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.062236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.062519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.067934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2ee61-f849-4df8-9925-f4d687c24740 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:21.102960 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:21.104180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:21.124845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.126168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.126713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.127297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.128694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:21.130378 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:21.132493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 83/169] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:20 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:21.133488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:21.249878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:21.254076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162840b-2874-4835-819f-c39c1f920e5a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:21.285658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 87/170] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:21.289963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:21.295015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.295911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.296423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.296948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.298356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:21.298894 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:21.300632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:21.311463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 84/171] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:21 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:21.326978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:21.471720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e6bba-7199-49dd-af75-3e4eccba63c4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:21.486249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:21.542624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.544214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.544713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.545186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.649915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:21.653341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:21.667821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d08038-b3cf-4aef-be9f-1d3fd23b7387 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:21.746470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.747289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.747573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.747843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.769877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-a97e7622-28b2-44e7-8d73-48343ba3b241 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:21.775900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:21.776902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:21.777701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 85/172] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:21.778896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984d6d-1928-44df-826c-20832be6b117 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.779083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984d6d-1928-44df-826c-20832be6b117 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.780191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83984d6d-1928-44df-826c-20832be6b117 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.780728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 86/173] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:21.786159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c75b3-e009-4678-a30a-4c6c9e79d656 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.786489 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c75b3-e009-4678-a30a-4c6c9e79d656 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.786753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 88/174] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:21.786914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c75b3-e009-4678-a30a-4c6c9e79d656 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.787657 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:21.788201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 87/175] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:21.789292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438cef5-399e-4f27-a657-e29918984446 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.789884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438cef5-399e-4f27-a657-e29918984446 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.789884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438cef5-399e-4f27-a657-e29918984446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.797749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:21.815297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-2438cef5-399e-4f27-a657-e29918984446 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:21.816939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-2438cef5-399e-4f27-a657-e29918984446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:21.848788 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:21.902783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865f98d-78c4-4f0e-a11b-66094cea6a18 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:21.903846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 88/176] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:21.905076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9e9-9b36-4cb9-8384-9614727a1e45 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.905373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9e9-9b36-4cb9-8384-9614727a1e45 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.905659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9e9-9b36-4cb9-8384-9614727a1e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.923519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ac29e9e9-9b36-4cb9-8384-9614727a1e45 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:21.924719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ac29e9e9-9b36-4cb9-8384-9614727a1e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:21.945862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:21.947378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:21.947861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:21.948390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:21.949374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:21.950106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:21.952319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:21.963616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 89/177] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:21.963616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.000821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 89/178] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:22.002224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f574d-6599-4747-a9fd-8cbf1b853343 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.002512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f574d-6599-4747-a9fd-8cbf1b853343 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.002795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f574d-6599-4747-a9fd-8cbf1b853343 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.004177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 90/179] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:22.004726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f574d-6599-4747-a9fd-8cbf1b853343 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.058319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.097204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908d020-d3a1-46a3-acb5-0f1ca6999dd0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.101559 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.124927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f574d-6599-4747-a9fd-8cbf1b853343 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.134269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.134946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.135198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.135475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.136652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.138137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.144821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 90/180] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:21 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:22.144821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.167996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.168639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.168918 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.169190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.180659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-02b70834-b47f-44f9-a74e-11abdd2596c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:22.185600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:22.186515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:22.187349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 91/181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:22.188808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.188876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.189129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.207309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:22.208467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:22.251773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191fdb-e438-4cf7-84ce-5e2ce2b6d706 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.280614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 92/182] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:22.281287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.292194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.293555 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.294440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.295067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] PATH_INFO: `/v3/users/64192fa2adce46b7a1ac80ee290def63` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.296731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] RBAC: Authorizing `identity:get_user(user_id=64192fa2adce46b7a1ac80ee290def63)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.299677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.302416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 91/183] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/users/64192fa2adce46b7a1ac80ee290def63 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:22.303328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.307739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.357488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.382954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d587cb-ebe1-4af2-b0a5-dd7e68fa17af None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.402411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f485c3-d102-463a-b84d-d3421cbcd8e3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.416419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.417053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.417330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.417599 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.427392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:22.432783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:22.433699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:22.434679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 93/184] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:22.435608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.441156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.441774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.442085 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.442359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.443148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:22.443453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.444863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd2b06-0159-403e-8848-e9a353a6d918 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.450679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 92/185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:22.451866 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a278-9431-4f29-8343-b5b3ad19a531 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.452154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a278-9431-4f29-8343-b5b3ad19a531 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.452429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a278-9431-4f29-8343-b5b3ad19a531 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.454349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 93/186] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:22.455025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a278-9431-4f29-8343-b5b3ad19a531 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.537620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.560535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fe1f3-5b26-4762-80fd-f19939857cd6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.576569 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a278-9431-4f29-8343-b5b3ad19a531 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.600429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.601053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.601327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.601607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.602369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:22.602680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.604054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.605787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.614377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 94/187] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:22.615151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.627047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.627788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.628078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.628356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.639308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:22.645375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:22.646329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:22.647121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:22.647745 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 94/188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:22.651187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.725033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b3312-7754-45bb-9e39-94ea700ae17c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.758608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.758803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e3cf-2209-401b-950e-01838b6ddd53 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:22.759217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.759496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.759770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.784955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.793095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:22.793716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.794044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.794325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.795044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:22.795354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.796743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1150d1-209c-4ad9-ac76-d0ca2d4eabb1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.804683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 95/189] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:22.805833 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:22.806151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:22.806435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:22.823872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:22.824668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:22.825395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:22.832980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:22.834506 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:22.892497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 96/190] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:22.893257 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:22.931882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 95/191] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:22.936458 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.001571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee640c-ad95-467e-8b8b-89efdb55755a None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.021457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.024746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.031119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c0ba9-0995-4703-9d2f-6fe53b4f20fb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.036283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.036921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.037201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.037478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] PATH_INFO: `/v3/users/48a55fdc97024ca18b3768649cfd8327` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.038657 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] RBAC: Authorizing `identity:get_user(user_id=48a55fdc97024ca18b3768649cfd8327)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.040090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.041678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 97/192] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/users/48a55fdc97024ca18b3768649cfd8327 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.042455 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.065394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.066085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.066365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.066639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.067349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:23.067653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.069006 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.076769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 96/193] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.077547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.144916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edea98-ff2c-4fa2-9cff-309757334b56 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.175773 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca9a2-83c7-4905-bd82-bf33d522cee4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.178807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.179436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.179713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.179986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.193535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.195004 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.196809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 98/194] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.197365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.209836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.210516 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.210799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.211079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.212223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.213649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.215253 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 97/195] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.218797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.242870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.246848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.298387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf043-caab-4de2-8ac8-dc3a4144c6ce None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.323482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1c001-a221-4a7c-8151-b72c84560bee None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.332062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.332703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.332988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.333273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] PATH_INFO: `/v3/projects/3a0d259190614f00af9efe9709ae53f6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.334472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] RBAC: Authorizing `identity:get_project(project_id=3a0d259190614f00af9efe9709ae53f6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.335923 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.337598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 99/196] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/projects/3a0d259190614f00af9efe9709ae53f6 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.338386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.357759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.358409 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.358690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.358962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.427721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.429098 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.435706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d457f7-3232-4b40-9757-0d454f388e05 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.450706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.455622 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.471933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.472563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.472840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.473113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.474290 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.475716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.477471 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 100/197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.478082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.533145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 98/198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:23.534097 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.577624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e63b4-45c6-4af4-8bb0-043fc3dc665d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.611409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.612281 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.612566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.612849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.630797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea1c6-57df-479b-b207-93e7129befbc None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.656140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.663444 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.668930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.669557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.669836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.670146 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.670824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.672362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.682853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.684338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.693405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 99/199] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:23 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:23.706170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.804017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1228c2e-8bbe-4a51-a48c-74d53253c76f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.811789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 101/200] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:23.814329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.851588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.852103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.852103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:23.852259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:23.853039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=3a0d259190614f00af9efe9709ae53f6, user.id=64192fa2adce46b7a1ac80ee290def63) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:23.854246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:23.856364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:23.864808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 100/201] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/role_assignments?user.id=64192fa2adce46b7a1ac80ee290def63&scope.project.id=3a0d259190614f00af9efe9709ae53f6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:23.865798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:23.915606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.939553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:23.944348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051501-7253-48d1-b21a-8f2d3a03a9a6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:23.998759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:23.999445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:23.999774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.000411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.026769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30232fc7-97ab-40d9-b302-40c75f274861 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.069273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.070086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.070372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.070647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.071838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.073269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.074966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 101/202] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:24.075804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:24.085109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.087027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.194356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:24.207475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab404c1b-62cd-4e47-a847-43133c31e2a2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.216053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 102/203] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:24.217086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:24.223451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:24.246490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.247143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.247424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.247699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.248964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.251241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.253353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 102/204] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:24 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:24.253916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:24.330098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181fba-5716-4dcd-8bcf-8f7220702276 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.365218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82745-3378-4bf6-8484-d2529d37b14d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.368535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.369178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.369463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.369741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.370870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6, user.id=48a55fdc97024ca18b3768649cfd8327) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:24.371185 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.372913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.380972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 103/205] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:24 2022] GET /identity/v3/role_assignments?user.id=48a55fdc97024ca18b3768649cfd8327&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:24.381869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:24.402374 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.403025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.403310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.403678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.404942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.407000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.409359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 103/206] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:24 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:24.414779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:24.439211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:24.485142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:24.507964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d6fc4-acaf-444f-8330-367cae1144dd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.574194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.575259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.576030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.578084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.579795 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=1615de6bef6a47afad30e79e6736f3aa) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:24.580470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f65cc-bf57-4943-9812-695f267c4b63 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:24.581000 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.583465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53ab0b-2407-4b47-8ee1-64934bf903ea None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.594893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 104/207] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:24 2022] GET /identity/v3/role_assignments?user.id=1615de6bef6a47afad30e79e6736f3aa&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:24.626025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f697551-f488-462b-9443-470724fbb220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:24.627019 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697551-f488-462b-9443-470724fbb220 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:24.627545 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697551-f488-462b-9443-470724fbb220 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:24.628015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f697551-f488-462b-9443-470724fbb220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:24.629096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f697551-f488-462b-9443-470724fbb220 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=1615de6bef6a47afad30e79e6736f3aa) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:24.629665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f697551-f488-462b-9443-470724fbb220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:24.631383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f697551-f488-462b-9443-470724fbb220 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:24.638313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 104/208] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:24 2022] GET /identity/v3/role_assignments?user.id=1615de6bef6a47afad30e79e6736f3aa&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:26.042733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f96ce-66f7-44dc-960e-8eefe8b86ea2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.043220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f96ce-66f7-44dc-960e-8eefe8b86ea2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.043220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f96ce-66f7-44dc-960e-8eefe8b86ea2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.044818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 105/209] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:26.050225 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.050633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.050803 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.069974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:26.071185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:26.134149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030f09-4413-45f7-a000-22ad4de92ae3 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.134662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030f09-4413-45f7-a000-22ad4de92ae3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.135084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030f09-4413-45f7-a000-22ad4de92ae3 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.136813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 106/210] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:26.142685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.143242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.143723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.146046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 105/211] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:26.162928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:26.164525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:26.165693 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.214302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.249643 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 107/212] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:26.275932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b903-5e65-4b0a-9d4e-8729b4834d6b None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:26.278600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.316177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595152-9650-49d7-9764-820162264081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:26.317157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595152-9650-49d7-9764-820162264081 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.317646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595152-9650-49d7-9764-820162264081 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.318166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595152-9650-49d7-9764-820162264081 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.319551 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7595152-9650-49d7-9764-820162264081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:26.321197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7595152-9650-49d7-9764-820162264081 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:26.323357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 106/213] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:26.329024 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595152-9650-49d7-9764-820162264081 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.334300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.397982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c72d6-274d-4c3f-b9a1-f263f55f419b None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:26.439995 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:26.441138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.441618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.442146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.453913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:26.454310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595152-9650-49d7-9764-820162264081 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:26.459781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:26.460661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:26.461438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:26.464200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 108/214] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:26.467823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.471776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.494408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:26.495288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.495583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.495860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.496557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:26.498639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:26.526856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 107/215] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:26 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:26.603239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.606950 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65346f88-4f9c-4b90-8980-fa599071a195 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:26.647365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:26.648060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.648349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.648635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.649434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:26.649751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:26.651217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b578148-d300-4b12-b40c-ef9bff0a5fc9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:26.661620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 109/216] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:26.670987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.686179 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6346e-a748-4ae4-acb6-edb10054ee81 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.686467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6346e-a748-4ae4-acb6-edb10054ee81 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.686915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6346e-a748-4ae4-acb6-edb10054ee81 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.688529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 110/217] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:26.694090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a774-7a67-4a4a-a2b4-6415200f15fc None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.694573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a774-7a67-4a4a-a2b4-6415200f15fc None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.695374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a774-7a67-4a4a-a2b4-6415200f15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.714627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6ad4a774-7a67-4a4a-a2b4-6415200f15fc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:26.715816 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6ad4a774-7a67-4a4a-a2b4-6415200f15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:26.800481 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ee138-dfbb-479e-9c9f-54c95840642d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:26.803041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 111/218] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:26.804551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee88a-5e51-4942-a784-a63bf18edbf4 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.805047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee88a-5e51-4942-a784-a63bf18edbf4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.805530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee88a-5e51-4942-a784-a63bf18edbf4 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.811168 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 112/219] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:26.817071 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.817350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.817638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.838712 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:26.841979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:26.842099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:26.842099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:26.842312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:26.842428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] PATH_INFO: `/v3/users/eaca5003069d41028d2a2f2987908632` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:26.866618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] RBAC: Authorizing `identity:get_user(user_id=eaca5003069d41028d2a2f2987908632)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:26.868606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:26.870628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 108/220] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/users/eaca5003069d41028d2a2f2987908632 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:26.871235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.901561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.929273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 113/221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:26.930166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:26.957542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:26.985669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eced3-1e6c-44bf-91fe-9384439efd48 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.023809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.024808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.025324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.025783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.037794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:27.043928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:27.044823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:27.045667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:27.046332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf99558-de52-45f3-ba04-a9b741fea480 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.048890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 109/222] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:27.049896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.086580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.087505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.087970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.088428 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.156956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.162216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.163995 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.175338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0f87-e183-4a1a-b0f0-adb18ead0a83 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.190032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.221088 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.221750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.222076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.222362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.236549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:27.244723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:27.245656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:27.246517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:27.251407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 110/223] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:27.252313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.284661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 114/224] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:27.285824 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.377814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa8d5-375e-4e49-b5a9-e69020c2429c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.415105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.415750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.416035 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.416307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.417031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:27.417334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.418275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e30ac-e2a8-487f-8e4b-3ccb995249d1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.418734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.427303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 111/225] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.428141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.447249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.456112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.472111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.472760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.473037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.473306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.474202 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:27.474561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.476607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.487886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 115/226] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.488713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.570458 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba4f0-d81b-4252-82c7-2712d70ca0e7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.611289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe373a6-53de-4ef4-bcd7-00428c203579 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.617661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.619028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.619525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.620084 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.621883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.624191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.627263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 112/227] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.628497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.662754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.663563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.663748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.664033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.665303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.666914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.668640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 116/228] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.669316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.727524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.730074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:27.757518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9be2e-f245-46cb-82b8-5f8eba01db46 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.776174 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737645-e10b-456f-9598-f0aa12f4a287 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.799413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.800431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.800961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.801438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.803051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.804725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.807137 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 113/229] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.808386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.810753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.811502 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.811937 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.812190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.813240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=eaca5003069d41028d2a2f2987908632) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:27.814288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.815154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.830154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 117/230] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/role_assignments?user.id=eaca5003069d41028d2a2f2987908632&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=4372496541ff48858e12197c1d6b1e71 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.831184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:27.929461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd037f-90f9-431d-a00a-3b296fea5d9c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:27.970717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:27.971867 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:27.972335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:27.972903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:27.974503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:27.976257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:27.978407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 114/231] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:27.985701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:28.008109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060a433-db18-494c-ae67-d0cd00dc8fa0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:28.010647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:28.049864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:28.070423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:28.071077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:28.071358 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:28.071649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:28.079009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:28.080473 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:28.083363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 118/232] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:27 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:28.089626 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:28.186532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201c21-8222-43a0-8730-9d31d7bd8aa1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:28.222062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:28.222743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:28.223027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:28.223304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/1615de6bef6a47afad30e79e6736f3aa/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:28.225690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, user_id=1615de6bef6a47afad30e79e6736f3aa, role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:28.227306 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f035af-8b78-4df7-824b-1ac7eec25547 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:28.243121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 115/233] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:27 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/1615de6bef6a47afad30e79e6736f3aa/roles/150d91154f9544c7a914d19f74d59fe8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:28.287012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621f9b5-06b7-4aac-a1c8-d1453e502225 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:28.332946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:28.333916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:28.334377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:28.334794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/1615de6bef6a47afad30e79e6736f3aa/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:28.339475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, user_id=1615de6bef6a47afad30e79e6736f3aa, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:28.341214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72236d7b-1de7-4d27-8df6-58a4e97eea9d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:28.367736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 119/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:28 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/1615de6bef6a47afad30e79e6736f3aa/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:28.968336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f06a-5d50-471e-8eb7-b9d44dd93232 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:28.969057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f06a-5d50-471e-8eb7-b9d44dd93232 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:28.969348 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f06a-5d50-471e-8eb7-b9d44dd93232 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:28.970869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 116/235] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:28.980422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:28.980716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:28.980999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.002773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:29.003976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:29.033524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7dd-3eb9-40df-9c85-24bf6f4e1e08 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.033861 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7dd-3eb9-40df-9c85-24bf6f4e1e08 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.034107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7dd-3eb9-40df-9c85-24bf6f4e1e08 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.035457 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 117/236] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:29.049275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.049557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.049830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.075955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:29.077168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:29.088105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.142854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 120/237] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:29.197854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 118/238] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:29.199338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.221536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.223789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.362524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1847e-9df9-4d63-b7b6-fe0605f0c4af None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:29.375507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5f9e9-0a28-41c1-ab1a-e912332445ce None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:29.399105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:29.399841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.400130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.400408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.401651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:29.403230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:29.405290 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 121/239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:29.412498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.414575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:29.415230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.415512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.415793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.427146 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:29.432105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:29.432971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:29.433767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 119/240] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:29.439259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.506405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.521395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.543961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a662491-2ce3-491c-84b0-eb6b29e4427f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:29.580475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:29.581192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.581663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.582030 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.582819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:29.583198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:29.584867 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:29.602301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 120/241] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:29.606332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.642878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ead3f-03ac-43aa-bd87-71304ba8342d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:29.709065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:29.709701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.710006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.710287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.710977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:29.712420 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:29.756156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 122/242] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:29 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:29.793136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75a5dd-61ae-41c7-8c61-b131a0b4b59f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:29.819075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.832735 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:29.833407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:29.833688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:29.833980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] PATH_INFO: `/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:29.850735 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] RBAC: Authorizing `identity:get_user(user_id=c262bc88bc8740e4b5dfb4acea15c5b4)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:29.852298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8bc24-4ebc-4d65-9739-72fc35229b19 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:29.854451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 121/243] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:29 2022] GET /identity/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:29.863228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:29.979159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:29.982998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9cdc-cb8e-4c43-84f3-bcd4c2a92d36 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.022220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.023089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.023515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.023929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.095114 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:30.096708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:30.189102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4502-e4fa-43a6-a223-c6f40ccf9079 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.189734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4502-e4fa-43a6-a223-c6f40ccf9079 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.190394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4502-e4fa-43a6-a223-c6f40ccf9079 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.196625 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 122/244] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:30.197972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.198513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.198513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.204610 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:30.210652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 123/245] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:30.217367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.224354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:30.225825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:30.315751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 123/246] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:30.346554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.348995 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b642d-e5ba-4856-8a31-5ce1d68557b9 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.396817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.397462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.397739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.398070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.399238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:30.400685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:30.403124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 124/247] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:30.409345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.476492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb9677-e0a2-45cd-8365-9c73affe78fb None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.491909 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:30.502642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:30.519353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.520002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.520281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.520610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.533274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:30.538425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:30.539307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:30.544887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 124/248] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:30.551339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.557421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0443-d407-4c86-9b39-df15c015bf36 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.601748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.602467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.602842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.603216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.604108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=c262bc88bc8740e4b5dfb4acea15c5b4) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:30.604534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:30.606102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:30.613275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 125/249] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/role_assignments?user.id=c262bc88bc8740e4b5dfb4acea15c5b4&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:30.672532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf618-dd48-48bf-b03b-2c5cc3dd7ce5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.712356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.712998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.713289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.713570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.714331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:30.714643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:30.716053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12848d-269f-45b5-b647-abc765c62052 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:30.726302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 125/250] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:30.732001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.738032 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486614f-0f7e-4683-b174-49ecb43022bd None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.738328 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486614f-0f7e-4683-b174-49ecb43022bd None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.738596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486614f-0f7e-4683-b174-49ecb43022bd None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.741223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 126/251] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:30.752981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.753347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.753563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.774698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:30.776334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:30.810833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:30.888162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 127/252] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:30.907354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdfcb5-11b0-471d-bd08-dc565f43ac2f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:30.922866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:30.959166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:30.970822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:30.971809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:30.972309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:30.972775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] PATH_INFO: `/v3/users/eaca5003069d41028d2a2f2987908632` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:30.976246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] RBAC: Authorizing `identity:get_user(user_id=eaca5003069d41028d2a2f2987908632)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:30.978028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:30.989032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 126/253] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/users/eaca5003069d41028d2a2f2987908632 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:30.990279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.103965 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80897ca8-5771-4c0c-b89b-e1dfa650abbb None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.158520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.159095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.159377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.160025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.173876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-61ec0408-373f-49c6-b759-a304ea3a78f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:31.179334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:31.180198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:31.180987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 128/254] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:31.182220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bdc1e-8b2c-46c4-b397-73b9e22ff923 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.182570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bdc1e-8b2c-46c4-b397-73b9e22ff923 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.182852 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bdc1e-8b2c-46c4-b397-73b9e22ff923 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.184447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 129/255] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:31.190512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bdc1e-8b2c-46c4-b397-73b9e22ff923 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.217130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:31.237419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebea7-4988-4f73-92bf-8eff06f0d653 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.305303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.305977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.306264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.306538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.308772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:31.310241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc646f14-720a-4de5-96de-37481ce4e980 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:31.311926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 127/256] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:30 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:31.315878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.315878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.315878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.352027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:31.353222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:31.375879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bdc1e-8b2c-46c4-b397-73b9e22ff923 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.413905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.414969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.415649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.417095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.418083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:31.418617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:31.420212 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:31.431248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 130/257] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:31.432794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.469750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:31.483599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 128/258] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:31.486292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.606942 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e322ea-1db1-4c80-8d54-c325cfb46dfe None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.613125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:31.635199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8489b18-e34f-4a8a-a819-2822bf195c36 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.692127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.692700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.692977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.693251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.695343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:31.696803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:31.698689 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 129/259] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:31 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:31.699598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.746644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.747326 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.747613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.747897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/eaca5003069d41028d2a2f2987908632/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.758747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, user_id=eaca5003069d41028d2a2f2987908632, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:31.760212 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:31.767918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:31.785220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 131/260] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:31 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/eaca5003069d41028d2a2f2987908632/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:31.786373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:31.824760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46c97f-df7b-40e9-8679-0af58f70039c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:31.927266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82a22-7472-410e-a980-153481649e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:31.931195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82a22-7472-410e-a980-153481649e5e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:31.931759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82a22-7472-410e-a980-153481649e5e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:31.932260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82a22-7472-410e-a980-153481649e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:31.959223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de82a22-7472-410e-a980-153481649e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-7de82a22-7472-410e-a980-153481649e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:31.966804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:31.967853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:31.969524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:31.971347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 130/261] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:31.983253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82a22-7472-410e-a980-153481649e5e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:32.049095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5321c-3e0c-4f6b-89c2-e2ea95c5d46d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.099880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.100855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.101355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.101837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.105595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82a22-7472-410e-a980-153481649e5e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.122288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:32.135677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:32.149788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.149788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.150249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.150412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.151321 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:32.151633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.153882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:32.164900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 131/262] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:32.171221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:32.220133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.222292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:32.398087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 132/263] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:32.404257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:32.430201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542016-9040-4ff3-a588-c620ac2a59be None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.478975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:32.536254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.538274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.538939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.540139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] PATH_INFO: `/v3/users/1615de6bef6a47afad30e79e6736f3aa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.542215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] RBAC: Authorizing `identity:get_user(user_id=1615de6bef6a47afad30e79e6736f3aa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.544201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b518-3702-4c74-85c9-33c3b27675f9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.545910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:32.554711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 132/264] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:32 2022] GET /identity/v3/users/1615de6bef6a47afad30e79e6736f3aa => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:32.556341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:32.609483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.610602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.611494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.612356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.613738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.615386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:32.617642 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 133/265] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:32 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:32.624406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:32.705045 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:32.805341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa591e2-c42d-4432-83a7-1c0f75c30c4e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.808198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40587f93-0108-4700-a1fe-94da79d5f249 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:32.840991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:32.879898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.880534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.880818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.881097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.883834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:32.884450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:32.884728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:32.885003 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:32.885799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=1615de6bef6a47afad30e79e6736f3aa) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:32.886142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.887592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be340f-23e8-4162-8de1-86b1ed02ff75 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:32.901321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 134/266] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:32 2022] GET /identity/v3/role_assignments?user.id=1615de6bef6a47afad30e79e6736f3aa&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:32.987670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:32.989153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:33.032620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfc30b-e0b3-4fa0-9311-dea38c159853 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.032969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfc30b-e0b3-4fa0-9311-dea38c159853 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.033186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfc30b-e0b3-4fa0-9311-dea38c159853 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.035863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 135/267] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:33.041986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.041986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.041986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.072754 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:33.072926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.079856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:33.177006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 133/268] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:32:33.183040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.213967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 136/269] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:33.237694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.323096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee30ca-2ac3-44bc-ad28-89bc8e86cf45 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.366551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.366869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b705c-c64c-4e59-85ad-232504419197 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.367731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.368366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.368851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.371063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:33.372719 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d9129-5a02-4fbe-a572-f2e36c6589df None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:33.375692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 134/270] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:33.377238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429b6c6-b8f4-4c0b-ae6a-30915fe54bd3 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.377526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429b6c6-b8f4-4c0b-ae6a-30915fe54bd3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.377800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429b6c6-b8f4-4c0b-ae6a-30915fe54bd3 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.379235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.382476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 135/271] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:33.383763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ef2c-ab8b-473b-9543-d817fb2f7fb9 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.384049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ef2c-ab8b-473b-9543-d817fb2f7fb9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.384413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ef2c-ab8b-473b-9543-d817fb2f7fb9 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.386170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 136/272] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:33.388442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ef2c-ab8b-473b-9543-d817fb2f7fb9 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.409151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.409968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.410244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.410515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.417635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.421318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c15a6074-ad1e-4cbc-889d-99c506c60aa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:33.426935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:33.427988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 137/273] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:33.433836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f93aac-2fea-4643-b793-058e161de335 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.434659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f93aac-2fea-4643-b793-058e161de335 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.434659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f93aac-2fea-4643-b793-058e161de335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.447515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-e7f93aac-2fea-4643-b793-058e161de335 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:33.448687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e7f93aac-2fea-4643-b793-058e161de335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:33.507878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ef2c-ab8b-473b-9543-d817fb2f7fb9 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.529960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 138/274] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:33.529960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.529960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.529960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.546673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:33.548165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:33.553234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.553931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.554314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.554655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.555525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=1615de6bef6a47afad30e79e6736f3aa) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:33.555838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:33.557264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:33.563976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 137/275] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/role_assignments?user.id=1615de6bef6a47afad30e79e6736f3aa&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:33.564656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.654469 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 139/276] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:33.655559 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.672464 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.690110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac76f94-7db1-4934-90b4-1febeae8d577 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.702287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.732283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.733282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.733786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.734299 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.735400 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:33.735935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:33.737873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:33.749699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 138/277] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:33.750878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.801754 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa7b98-0718-484e-8722-8120a5974a61 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.843996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.844800 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.845181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.845469 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.847279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:33.848723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:33.850791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 140/278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:33.851346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.874723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61ebe0-24c3-4dad-b51c-7c5ce375c729 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:33.910376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:33.911043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:33.911354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:33.911634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:33.922521 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:33.927900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:33.928793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:33.929594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 139/279] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:33.930765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:33.943302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.958453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:33.966717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e4b2-7950-471c-a992-b6d521e10a07 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.005498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.006168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.006450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.006726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] PATH_INFO: `/v3/users/d23deffa3cdb42279ca77bb853d11fb7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.021017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] RBAC: Authorizing `identity:get_user(user_id=d23deffa3cdb42279ca77bb853d11fb7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.022495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.028805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 141/280] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:33 2022] GET /identity/v3/users/d23deffa3cdb42279ca77bb853d11fb7 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:34.028805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.042853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeda39b-c547-4d04-98f2-7fc6ec40e8e4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.083661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.084295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.084829 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.085110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.086240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.088193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.114978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 140/281] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:33 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:34.115887 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.152368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c9260-56bc-40fa-b7f4-2a28b60803a0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.194315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:34.209166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.210214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.210679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.211150 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.212069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:34.212559 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.214181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.227927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 142/282] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:34.235526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.252644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195ecd-c201-49e6-ba64-915fc71534bf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.261707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:34.305793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.306451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.306735 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.307013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.307817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, user.id=d23deffa3cdb42279ca77bb853d11fb7, scope.system=all) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:34.308132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.309608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.319750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 141/283] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 06:32:34 2022] GET /identity/v3/role_assignments?user.id=d23deffa3cdb42279ca77bb853d11fb7&scope.system=all&role.id=4372496541ff48858e12197c1d6b1e71 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:34.375651 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260220c6-3169-4ebc-ada7-cef41eb5510a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.416015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.416768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.417054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.417337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] PATH_INFO: `/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.419229 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] RBAC: Authorizing `identity:get_user(user_id=c262bc88bc8740e4b5dfb4acea15c5b4)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.420719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.423692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 143/284] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:34 2022] GET /identity/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:34.429759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.551104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c1ef9-ca53-44c0-8658-3b9b7c08bca7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.579537 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:34.589241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.589894 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.590208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.590488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.591692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.593194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb552a-4d48-4e1a-b1ef-c92ee9ea3ca1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.599633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 142/285] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:34 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:34.601025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.647377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f81db4-ad88-4b39-ad09-46de7464fe23 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.647698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f81db4-ad88-4b39-ad09-46de7464fe23 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.647936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f81db4-ad88-4b39-ad09-46de7464fe23 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.651799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 143/286] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:34.660961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.660961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.660961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.688242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:34.689487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:34.748371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:34.780123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1b2ec-4899-49bf-9b33-4ff7dfeaed9e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:34.815605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 144/287] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:34.841695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:34.855727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:34.856417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:34.856703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:34.856984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/c262bc88bc8740e4b5dfb4acea15c5b4/roles/9838df12a6f345f0a80ce47deaca2479` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:34.861819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, user_id=c262bc88bc8740e4b5dfb4acea15c5b4, role_id=9838df12a6f345f0a80ce47deaca2479)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:34.863456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:34.889836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 144/288] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:32:34 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/users/c262bc88bc8740e4b5dfb4acea15c5b4/roles/9838df12a6f345f0a80ce47deaca2479 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:32:34.977718 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232796f-3143-4073-8424-e5721bf0c933 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:35.007791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:35.022166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:35.022855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:35.023150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:35.023426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:35.044926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:35.050377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:35.051318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:35.052124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:35.054294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 145/289] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:35.059232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:35.209446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b441-8452-4cfb-8362-cf5f1916b464 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:35.249562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:35.250270 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:35.250557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:35.250869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:35.251604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:35.251914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:35.253273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:35.266836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 145/290] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:35.272738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:35.388138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbe4ed-1b9a-4d72-a179-18540f24be72 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:35.428876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:35.429528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:35.429812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:35.430117 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] PATH_INFO: `/v3/users/eaca5003069d41028d2a2f2987908632` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:35.431311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] RBAC: Authorizing `identity:get_user(user_id=eaca5003069d41028d2a2f2987908632)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:35.432746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:35.434736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 146/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:35 2022] GET /identity/v3/users/eaca5003069d41028d2a2f2987908632 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:35.440187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:35.449802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:35.566451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a907c-57dc-4878-a5d1-d6309da1d68d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:35.614663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:35.615299 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:35.615583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:35.615864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:35.701077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:35.702731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:35.716740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:35.814407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 146/292] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:35.823059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:35.893660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:35.947644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c4b64-ffad-4f33-9e8b-6dc6a46f92db None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:35.989506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:35.990173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:35.990461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:35.990737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:35.991912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:35.993353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7dc7-3c6b-4343-9fb8-91a85a79d3f1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:35.995039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 147/293] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:35 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:36.000957 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:36.137201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df014466-a206-48e0-8df6-7a22072f7f29 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:36.148442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:36.191510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:36.192540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.193016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.193484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.194504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=eaca5003069d41028d2a2f2987908632) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:36.195012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:36.196839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616d9c-c554-434b-8348-4f34c0362310 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:36.207031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 147/294] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:35 2022] GET /identity/v3/role_assignments?user.id=eaca5003069d41028d2a2f2987908632&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=4372496541ff48858e12197c1d6b1e71 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:36.677218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbbfe-e1a0-4398-9157-24e24cbf966f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.677522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbbfe-e1a0-4398-9157-24e24cbf966f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.677788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cbbfe-e1a0-4398-9157-24e24cbf966f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.679200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 148/295] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:36.685435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.686313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.686857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.708274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:36.709768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:36.812394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 148/296] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:36.834749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abb4f-2227-42e3-bcf7-54c9b3133d77 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.835245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abb4f-2227-42e3-bcf7-54c9b3133d77 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.835318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8abb4f-2227-42e3-bcf7-54c9b3133d77 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.835757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:36.836786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 149/297] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:36.842028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412b91-cfb3-4a44-a75a-c939376985ed None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.842551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412b91-cfb3-4a44-a75a-c939376985ed None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.843091 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412b91-cfb3-4a44-a75a-c939376985ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.865078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-ee412b91-cfb3-4a44-a75a-c939376985ed None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:36.866474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ee412b91-cfb3-4a44-a75a-c939376985ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:36.940806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:36.942494 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 150/298] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:36.943946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7efbc-238f-4e1f-8040-184e714c65b5 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.944303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7efbc-238f-4e1f-8040-184e714c65b5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.944649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7efbc-238f-4e1f-8040-184e714c65b5 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.946101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 151/299] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:36.951506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.951790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.952064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:36.955318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e18875-8951-492d-8b7f-611c81b1acce None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:36.974013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:36.975197 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:36.995556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:36.996283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:36.996632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:36.996976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.008864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:37.013847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:37.014946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:37.015851 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 149/300] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:37.019413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.025689 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.050107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 152/301] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:37.050107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.128627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d36668-3897-4431-b118-f4619a12c604 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.162708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82251-dc79-4538-b007-2ee52fac67b8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.164633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.165266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.165546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.165826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.167769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.169209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.170825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 150/302] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.172757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.201522 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.203294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.203898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.204226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.204451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.205209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:37.205506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.206885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.215461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 153/303] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.216057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.268103 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.282719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55090-7475-47aa-93e4-7f84ffb69994 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.319809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.320455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.320739 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.321062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.326402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70441d3-d79c-482f-b5c1-0e6e53988be8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.332677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:37.337668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:37.338732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:37.339524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 151/304] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:37.340257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.375289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.375916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.376198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.376474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] PATH_INFO: `/v3/groups` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.377133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.378596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.396575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 154/305] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 06:32:37 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:37.397485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.448318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.465008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ea26-e6a3-4e91-b817-d8f3ce4ca68d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.508879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.509887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.510370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.510834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] PATH_INFO: `/v3/users/c9c1b892e0a74c10818068c88831d5ab` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.527324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] RBAC: Authorizing `identity:get_user(user_id=c9c1b892e0a74c10818068c88831d5ab)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.528817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02126ce-7a2a-403c-a64a-109f6ce1819b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.529452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e938c3-85cb-4e45-9db0-b986ef026fef None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.536058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 152/306] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/users/c9c1b892e0a74c10818068c88831d5ab => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.536058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1c55-b2b3-431d-964e-c5106920dc7c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.536058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1c55-b2b3-431d-964e-c5106920dc7c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.536058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1c55-b2b3-431d-964e-c5106920dc7c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.536058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 153/307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:37.539243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.539696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d1c55-b2b3-431d-964e-c5106920dc7c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.577042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.578402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.578771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.579881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.580612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:37.580931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.582954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99463d9-d4d1-43fc-a6f8-76b48e7a6571 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.591886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 155/308] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.593717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.594335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.594629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.615111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:37.616644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:37.653657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d1c55-b2b3-431d-964e-c5106920dc7c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.689980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 156/309] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:37.690845 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.703837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.705257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.705738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.706521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.708063 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, user.id=c9c1b892e0a74c10818068c88831d5ab, scope.system=all) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:37.708526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.710318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.716925 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 154/310] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/role_assignments?user.id=c9c1b892e0a74c10818068c88831d5ab&scope.system=all&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.718511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.736274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.797224 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:37.804307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dd24e-e48d-44ca-85dc-48f29231be99 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.854269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.854943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5b251-a603-4365-985a-3eea221662e6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:37.855041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.855195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.855468 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] PATH_INFO: `/v3/users/d23deffa3cdb42279ca77bb853d11fb7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.856978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] RBAC: Authorizing `identity:get_user(user_id=d23deffa3cdb42279ca77bb853d11fb7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:37.859751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:37.862124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 157/311] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/users/d23deffa3cdb42279ca77bb853d11fb7 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:37.867440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:37.909346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:37.910087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:37.910371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:37.910664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:37.924238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:37.930923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:37.931825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:37.932647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 155/312] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:37.940242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:38.003658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab598-0888-46dd-8eb9-51e3c622de60 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.015991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:38.050874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119730e-7187-4a24-8947-b63061747f30 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.052705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.053642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.053925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.054230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] PATH_INFO: `/v3/system/users/d23deffa3cdb42279ca77bb853d11fb7/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.056683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d23deffa3cdb42279ca77bb853d11fb7, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.058167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.078144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 158/313] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 06:32:37 2022] PUT /identity/v3/system/users/d23deffa3cdb42279ca77bb853d11fb7/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:32:38.094751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.095398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.095743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.096012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.096834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:38.097152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.098590 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.109532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 156/314] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:38.116107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:38.151637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:38.236072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8c88-ce0a-403e-9281-ba29af1f6d01 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.273309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.274328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.274808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.275296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] PATH_INFO: `/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.276669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] RBAC: Authorizing `identity:get_user(user_id=c262bc88bc8740e4b5dfb4acea15c5b4)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.278304 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.281068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 159/315] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:38 2022] GET /identity/v3/users/c262bc88bc8740e4b5dfb4acea15c5b4 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:38.286925 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:38.393224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253dc8-becf-49a8-87d4-250e4e0eed8f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.427854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.428823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.429299 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.429785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.434138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:38.509239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.510761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.619572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 157/316] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:38.627079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:38.738316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:38.741986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d595d8-c30f-44e3-98ad-d0f1ef8d3986 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.777256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.777904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.778214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.778493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.779658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.781752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bba4ae-a62f-4b8d-aebf-71e80235f194 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.783791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 160/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:38 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:38.789525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:38.821689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:38.899465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e0560-dc87-45e8-bd4f-30b28af165f8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:38.935526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:38.936471 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:38.937523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:38.938023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:38.939030 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7, user.id=c262bc88bc8740e4b5dfb4acea15c5b4) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:38.939551 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:38.941161 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7eb60-2e52-4ffc-a607-5fb941e6f107 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:38.947466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 158/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:32:38 2022] GET /identity/v3/role_assignments?user.id=c262bc88bc8740e4b5dfb4acea15c5b4&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:39.581686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715c355-26ce-452d-9639-0599bcff34f8 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:39.582264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715c355-26ce-452d-9639-0599bcff34f8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:39.582264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715c355-26ce-452d-9639-0599bcff34f8 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:39.583640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 161/319] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:39.588752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1218349-d0d2-4b27-ab66-bf7da9f87532 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:39.589051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1218349-d0d2-4b27-ab66-bf7da9f87532 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:39.589334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1218349-d0d2-4b27-ab66-bf7da9f87532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:39.607854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-a1218349-d0d2-4b27-ab66-bf7da9f87532 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:39.609580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a1218349-d0d2-4b27-ab66-bf7da9f87532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:39.682185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 159/320] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:39.705287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715c355-26ce-452d-9639-0599bcff34f8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:39.762351 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922cb8f-ce2a-4148-92ca-3cd3d06cf6d9 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:39.762655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922cb8f-ce2a-4148-92ca-3cd3d06cf6d9 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:39.762908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922cb8f-ce2a-4148-92ca-3cd3d06cf6d9 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:39.764387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 160/321] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:39.770415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:39.770978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:39.771275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:39.795373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:39.797781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:39.816271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715c355-26ce-452d-9639-0599bcff34f8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:39.823628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:39.852211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:39.852778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:39.853065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:39.853339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:39.854522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:39.855965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:39.857737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 162/322] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:39.863759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:39.887184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 161/323] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:39.896803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:39.908547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:39.972077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ce184-d5f1-42a7-a1bf-d19bde76e243 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.011677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.012291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.012564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.012836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] PATH_INFO: `/v3/groups` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.013492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.014091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bf4d8-8ed8-4682-b05b-401c702bfb42 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.014998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.033237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 163/324] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jul 27 06:32:39 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:32:40.063255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.063936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.064220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.064498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.076632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-f88f43ef-3007-4815-9dd5-34119fc26d26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:40.082566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:40.083583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:40.084418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 162/325] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:40.089972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.174060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205793-ce6d-4c69-bcd7-23b51a16ace0 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.174324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205793-ce6d-4c69-bcd7-23b51a16ace0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.174700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61205793-ce6d-4c69-bcd7-23b51a16ace0 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.176348 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 163/326] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:40.181727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.182041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.184076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.200266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:40.203321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:40.204516 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:40.212018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1dd5-860b-4530-a354-75dd5d3fc906 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.218861 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:40.255913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.256540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.256829 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.257106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.257826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:40.258191 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.259563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.268239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 164/327] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:40.274085 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.292812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 164/328] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:40.316445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.393749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8bf04-af2d-477e-86b9-770e0df7f7ba None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.434384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.435044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.435411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.435761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] PATH_INFO: `/v3/users/c9c1b892e0a74c10818068c88831d5ab` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.437960 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] RBAC: Authorizing `identity:get_user(user_id=c9c1b892e0a74c10818068c88831d5ab)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.439611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.441575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 165/329] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/users/c9c1b892e0a74c10818068c88831d5ab => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:40.450927 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.452712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426fe30-3f97-4eb9-aa54-0945f7b1870f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.482801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:40.500542 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.501455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.501891 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.502368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.518296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:40.524446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:40.525556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:40.526595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:40.528891 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 165/330] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:40.534927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.544310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:40.585928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cf88e-d9e4-4834-82ed-4d3f536eac0b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.632972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.634167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.634981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.635441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] PATH_INFO: `/v3/system/users/c9c1b892e0a74c10818068c88831d5ab/roles/9838df12a6f345f0a80ce47deaca2479` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.637076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c9c1b892e0a74c10818068c88831d5ab, role_id=9838df12a6f345f0a80ce47deaca2479)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.639318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.655016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 166/331] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jul 27 06:32:40 2022] PUT /identity/v3/system/users/c9c1b892e0a74c10818068c88831d5ab/roles/9838df12a6f345f0a80ce47deaca2479 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:40.656810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad348-8421-4b13-a82b-904559c4e1c2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.702033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.703332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.703625 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.703906 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.704666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:40.705009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.706455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.716718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 166/332] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:40.723212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.846771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:40.865000 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea55b7-83b0-4eac-9fde-110670ada683 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:40.904181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:40.905079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:40.905600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:40.906104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] PATH_INFO: `/v3/users/d23deffa3cdb42279ca77bb853d11fb7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:40.907510 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] RBAC: Authorizing `identity:get_user(user_id=d23deffa3cdb42279ca77bb853d11fb7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:40.909095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba964432-633d-4b9a-bad8-71b83351ef3e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:40.911184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 167/333] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/users/d23deffa3cdb42279ca77bb853d11fb7 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:40.916778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:40.998281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:41.040955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d3d5a-fa4b-412b-921a-8e27268d87d8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:41.080387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:41.081257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:41.081687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:41.082172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:41.083149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] RBAC: Adding query filter params (role.id=4372496541ff48858e12197c1d6b1e71, user.id=d23deffa3cdb42279ca77bb853d11fb7, scope.system=all) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:41.083628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:41.085204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05cf3c-f274-4bea-9312-41bcd9937f7b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:41.101456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 167/334] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 06:32:40 2022] GET /identity/v3/role_assignments?user.id=d23deffa3cdb42279ca77bb853d11fb7&scope.system=all&role.id=4372496541ff48858e12197c1d6b1e71 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:42.518123 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c840-d9a1-434e-b89a-711f42d9ef58 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:42.518971 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c840-d9a1-434e-b89a-711f42d9ef58 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:42.519393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c840-d9a1-434e-b89a-711f42d9ef58 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:42.521148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 168/335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:42.526407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:42.527292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:42.527802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:42.555491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:42.557050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:42.636474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 168/336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:42.679462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c840-d9a1-434e-b89a-711f42d9ef58 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:42.811807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c840-d9a1-434e-b89a-711f42d9ef58 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:42.857416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:42.858096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:42.858924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:42.859381 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:42.875619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-07715037-57f3-4d0d-b698-66c444c4024e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:42.880692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:42.881575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:42.882403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:42.883476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 169/337] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:42.888125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:42.955163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a273-c419-45c3-bc35-b52aaf630895 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:42.955480 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a273-c419-45c3-bc35-b52aaf630895 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:42.955726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a273-c419-45c3-bc35-b52aaf630895 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:42.962039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 170/338] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:42.980289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:42.981173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:42.981412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:42.981819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.017976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:43.017976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:43.031759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:43.056716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3bd4-069d-470a-9265-a02c2c302dd3 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.115262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 171/339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:43.124944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.125812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.126136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.126416 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.127135 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:43.127443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:43.128813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:43.141457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.142155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 169/340] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:43.149002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.340855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab556a3-050b-4fbc-b185-07ae101743fa None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.341670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9276d-54ed-4194-95dc-65022b12ce51 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.373726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:43.400766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.401400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.401683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.402167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.406360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.407077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.407454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.407749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.429262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:43.434479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:43.435353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:43.436166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 172/341] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:43.443453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.450261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:43.496370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:43.497837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:43.632116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 170/342] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:32:43.640565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e542893-4d05-47a5-8aa8-d32a42726866 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.642973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.673750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:43.697396 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.698057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.698354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.698603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.699456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:43.699771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:43.701149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:43.709920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 173/343] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:43.715872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.824873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:43.836198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113314-02ff-4329-ac02-4fec4a924654 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.878787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdfabf-9c8f-4557-9860-df7e40fb83cf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:43.897467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.898073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.898354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.898629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.899801 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:43.901238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933a85a-420f-4bad-97b5-1cdd8edf7eb5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:43.907713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 171/344] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:43.907713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3310b-f1bc-42a4-901e-f6d131b0cc88 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.907713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3310b-f1bc-42a4-901e-f6d131b0cc88 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.907713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3310b-f1bc-42a4-901e-f6d131b0cc88 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:43.908073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 172/345] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:43.909615 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3310b-f1bc-42a4-901e-f6d131b0cc88 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:43.950245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:43.950890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:43.951173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:43.951443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.028573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.031178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8ca2c-215c-4f5e-968d-29e12551fe3e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.033365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.053711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3310b-f1bc-42a4-901e-f6d131b0cc88 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.090001 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.090653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.090930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.091202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.097757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.099350 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62e3ee-a7ad-4d1a-ab69-586f39853764 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.101455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 173/346] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:44.102749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.103032 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.103306 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.105776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.125490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:44.126941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:44.160365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 174/347] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:44.162029 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328eab-ef72-47a4-88d2-340dc9ed3306 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.162518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328eab-ef72-47a4-88d2-340dc9ed3306 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.162987 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328eab-ef72-47a4-88d2-340dc9ed3306 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.164681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 175/348] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:44.165756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328eab-ef72-47a4-88d2-340dc9ed3306 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.199743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 174/349] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:44.200607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.290293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.312251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e8eaa-f5ae-4c57-888c-4fdeea470740 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.316763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328eab-ef72-47a4-88d2-340dc9ed3306 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.347679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.348339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.348623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.348901 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.350101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.351773 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97fcd6-4870-4845-a136-473127348613 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.355138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 175/350] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:44.356333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.356618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.356896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.368308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.368932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.369277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.369561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.370387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:44.370696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.372133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.378604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:44.379080 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 176/351] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:44.379895 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.380005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:44.414479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.457126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 176/352] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:44.457823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.545417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93f008-f7e7-439e-9226-3761a0d2d3da None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.552700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.587063 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0f16a-8aad-4303-86ca-bca181b9ce92 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.593850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.595570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.595861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.596137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.618632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-338a1e39-2c7c-495c-9db1-a99ad65737b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:44.625606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:44.626629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 06:32:44.627439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:44.632487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 177/353] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:44.634350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839533dd-ece4-407d-87fc-0218040b71b7 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.634775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839533dd-ece4-407d-87fc-0218040b71b7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.635056 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839533dd-ece4-407d-87fc-0218040b71b7 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.637275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 178/354] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:44.638141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839533dd-ece4-407d-87fc-0218040b71b7 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.642721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.644343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.644625 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.644905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.646116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.647549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.650815 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 177/355] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:44.651621 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.756335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.773748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839533dd-ece4-407d-87fc-0218040b71b7 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.799713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ad8b-9487-4516-821b-8752bd37fa05 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:44.821603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.822634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.823154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.823636 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.836512 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:44.836979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:44.837608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.837890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.838199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.838916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:44.839225 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:44.840582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f5ad07c3-d568-428c-8c4e-ae05ac4c757d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:44.844694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:44.845602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:44.846441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:44.849185 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 179/356] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:44.849538 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 178/357] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:44.851315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:44.851822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:44.852339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:44.854457 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.868748 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:44.874755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:44.876274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:44.976286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 180/358] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:44.977192 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:44.984641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e4efa-b422-4548-88e0-3e5f2a3aa7f6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.022452 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.023467 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.023947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.024428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.025296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:45.025789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.027241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.035779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 179/359] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.036950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.079487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.106277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83d33-3115-491b-8b7c-8d3196cac7e0 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.121928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.161069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052fd9-aa0e-4d3d-9c0f-0186c4b5ba54 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.162183 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.162871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.163158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.163437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.164181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:45.164491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.165922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.178404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 181/360] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.182306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.207536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.208257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.209704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.210041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] PATH_INFO: `/v3/users/c9c1b892e0a74c10818068c88831d5ab` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.211418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] RBAC: Authorizing `identity:get_user(user_id=c9c1b892e0a74c10818068c88831d5ab)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.213019 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219022d5-60c4-4af6-a836-6087a54a89df None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.215018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 180/361] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/users/c9c1b892e0a74c10818068c88831d5ab => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.221601 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d7e5-f4b5-45f8-baa0-f8db8a007d82 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.221888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d7e5-f4b5-45f8-baa0-f8db8a007d82 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.222226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984d7e5-f4b5-45f8-baa0-f8db8a007d82 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.224881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 181/362] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:45.225729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984d7e5-f4b5-45f8-baa0-f8db8a007d82 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.320070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e510-9fc7-487d-bb0d-7f2b9912caa2 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.358753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984d7e5-f4b5-45f8-baa0-f8db8a007d82 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.363647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.364603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.365086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.365560 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.378875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:45.385406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:45.386399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:45.387232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:45.390042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 182/363] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:45.391038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.400451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.401018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.401637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.402120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.402223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.423997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.503648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.505554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.538174 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591f25-0aac-407a-a6b0-ecb16a2b0d49 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.577018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.577213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.577450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.577674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.578619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] RBAC: Adding query filter params (role.id=9838df12a6f345f0a80ce47deaca2479, user.id=c9c1b892e0a74c10818068c88831d5ab, scope.system=all) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:45.578943 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.580387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f875da-2779-408e-92e1-87a06006aa6d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.593748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 183/364] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/role_assignments?user.id=c9c1b892e0a74c10818068c88831d5ab&scope.system=all&role.id=9838df12a6f345f0a80ce47deaca2479 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.595036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.595329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.595609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.618746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:45.620044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:45.630767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 182/365] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:45.636650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.719004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.750236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:45.771410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 184/366] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:45.772170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.845134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee817a4d-af9c-4021-b5ff-e4a58181a4c7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.897780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.899716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.900487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.900969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.902394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:45.903199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.904772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.904944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96716c7-1f34-4a11-9d55-4ab1920f42e2 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:45.916140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 183/367] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.917610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:45.948887 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:45.949911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:45.950345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:45.950742 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:45.953047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:45.955697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:45.961964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 185/368] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:45.961964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.048879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5896359-da44-491c-b9d9-f658550ac192 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.051450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.053251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.086184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6aa89-d52b-41c7-9697-422428b054d6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.090933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.091582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.091868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.092148 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.103653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:46.110075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:46.110987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:46.112062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 184/369] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:46.112726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.129995 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.130741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.131025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.131302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.206817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.208486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.218135 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca3349-deb9-42fc-bc1e-2c4476fd3428 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.262744 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.263544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.264119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.264949 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.265964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.267909 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.270274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 185/370] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:46.271509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.282303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.315680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.337329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 186/371] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:46.338549 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.396165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5bbb7-202d-49ac-b9c5-28413213f33e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.433509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.434165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.434450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.434729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.435553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:46.435869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.437431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.447633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 186/372] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:46.448652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.457577 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06321eeb-99c7-47a2-a83e-31ced0af5a7f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.496786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.497822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.498319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.498868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.499945 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:46.500450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.502159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.508619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 187/373] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=4372496541ff48858e12197c1d6b1e71 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:46.509679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.550313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.552315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.564547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb4159-d220-44b0-abcf-9215e04b0534 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.612490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.613626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.614787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.615387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.618174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.620659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.623657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 187/374] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:46.631040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.715962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f4779-cd13-46dd-a52a-c8b62b292b29 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.757859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c904c1e-06c0-48c4-9800-3a509bd5e23a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:46.797629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.798752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.799196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.799630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.800963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.802597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.805021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 188/375] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:46.810610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:46.826700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:46.829495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:46.829495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:46.829495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:46.862444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.956523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:46.957911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:46.992736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:46.997043 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cacf-b33b-4e68-b4b2-e8e7804cdd3d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.061818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.062191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.062477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.062757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.063558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:47.063875 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:47.065319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb181c3-0519-414f-bd1d-873a4337e7c2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:47.073498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 189/376] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:47.075825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844845d-bbf2-4733-89d5-9480c3c61f1b None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.076124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844845d-bbf2-4733-89d5-9480c3c61f1b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.076408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844845d-bbf2-4733-89d5-9480c3c61f1b None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.078174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 190/377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:47.091710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.092033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.092588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.109802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 188/378] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:47.115738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.129808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:47.131075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:47.239845 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b78a-e3c0-437b-8c1c-e10461bfd321 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.260217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 191/379] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:47.266112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.277781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.278558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.278844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.279129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] PATH_INFO: `/v3/projects/0d277a3b91684c12b74997019b6d7be2` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.280387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] RBAC: Authorizing `identity:get_project(project_id=0d277a3b91684c12b74997019b6d7be2)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:47.281862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:47.283883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 189/380] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/projects/0d277a3b91684c12b74997019b6d7be2 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:47.289093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.289245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.322052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.438803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045ec90-0c4f-4050-b7a1-482f5a05e9ab None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.482200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.483565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.484553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.485256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.488250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0a47-ce93-45be-9d17-caf608cb4f85 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.502440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a7bf8ac1-45a1-4e50-a03e-148fc5ae9a54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:47.507908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:47.508812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:47.509621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 190/381] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:47.510790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4f48-347b-4e96-a32d-757097072078 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.510889 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4f48-347b-4e96-a32d-757097072078 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.511171 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4f48-347b-4e96-a32d-757097072078 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.515978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 191/382] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:47.517676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f4f48-347b-4e96-a32d-757097072078 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.532941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.533605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.533888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.534214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] PATH_INFO: `/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.540367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] RBAC: Authorizing `identity:get_group(group_id=4e7b4cb9038e4d15934a9b37637f58d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:47.541814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2d835-ff00-4377-9f7c-96d9a39f9904 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:47.544617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 192/383] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:47.545931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e110a-5552-4070-915e-61883880d24f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.546244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e110a-5552-4070-915e-61883880d24f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.546659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e110a-5552-4070-915e-61883880d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.568512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-206e110a-5552-4070-915e-61883880d24f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:47.569721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-206e110a-5552-4070-915e-61883880d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:47.594326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.670325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f4f48-347b-4e96-a32d-757097072078 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.671502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 193/384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:47.672636 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e110a-5552-4070-915e-61883880d24f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.706313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.729789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.730487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.730988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.731540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.732530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:47.733129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:47.734965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:47.745135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 192/385] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:47.746252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.806263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e110a-5552-4070-915e-61883880d24f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.850486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.851580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.851873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.852151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:47.852941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] RBAC: Adding query filter params (group.id=4e7b4cb9038e4d15934a9b37637f58d7, role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=0d277a3b91684c12b74997019b6d7be2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:47.853254 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:47.854979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:47.864204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 194/386] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/role_assignments?group.id=4e7b4cb9038e4d15934a9b37637f58d7&scope.project.id=0d277a3b91684c12b74997019b6d7be2&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:47.865340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:47.889641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049edb81-712f-4ee8-981d-55f8c85b2b65 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:47.916460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.973482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:47.978765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:47.979766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:47.980298 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:47.980781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.000302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726818-a13b-499d-9179-a962c87199ce None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.006430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:48.012578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:48.013514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:48.014404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:48.019348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 193/387] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:48.024317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.058983 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.060832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.062741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.063192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.064511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.067062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.069285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 195/388] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:47 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:48.075184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.198176 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc4cba-1575-456b-b14d-c554abe751f3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.211351 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be7640-7a40-4fbe-8db7-83460b443067 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.257656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.258862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.259653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.260282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.261817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.263483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.266937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 196/389] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:48 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:48.267064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.267658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.267950 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.268218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.268939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:48.269243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.273781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.277737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.288371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 194/390] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:48.296651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.307942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:48.404289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:48.414766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ad92-51aa-4ceb-9e74-f621f24373bc None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.461381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.462872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.463528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.464011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.466505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, group_id=cabcedf045dd4a8e9c7455c5e756dfc6, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.469479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.489162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 197/391] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 06:32:48 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:32:48.552181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45101f-da16-49f1-bb6e-0baacda88cc7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.604069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.604715 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.604992 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.605266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.606445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.608399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.610010 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 195/392] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:48 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:48.615945 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.752474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:48.784444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0cb00-6adf-4100-9570-2417a0be55d3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.823793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:48.824443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:48.824714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:48.824993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:48.827877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:48.829337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953e992-8345-4616-961a-3b8019f55438 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:48.831364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 198/393] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:48 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:48.837089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:48.959511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863435-2fac-4258-8fbe-6209b32fab2a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:48.960992 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:49.004516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:49.005204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.005488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.005767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.008032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd68a4366748e6a6e25223e07c10a6, group_id=cabcedf045dd4a8e9c7455c5e756dfc6, role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:49.009462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48be60-961d-478e-acda-cc57d52e78fd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:49.032627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 196/394] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 06:32:48 2022] PUT /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/150d91154f9544c7a914d19f74d59fe8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:49.487586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f9e-980d-475d-88e0-b496af45b116 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.487998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f9e-980d-475d-88e0-b496af45b116 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.488186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a9f9e-980d-475d-88e0-b496af45b116 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.489731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 199/395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:49.495696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108ecd-987c-4e50-b10d-d3cafb62412f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.495985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108ecd-987c-4e50-b10d-d3cafb62412f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.496262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108ecd-987c-4e50-b10d-d3cafb62412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.516007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-68108ecd-987c-4e50-b10d-d3cafb62412f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:49.517209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-68108ecd-987c-4e50-b10d-d3cafb62412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:49.599460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82621901-41b8-45e7-876c-3e8726718878 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.599779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82621901-41b8-45e7-876c-3e8726718878 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.600018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82621901-41b8-45e7-876c-3e8726718878 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.600543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 197/396] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:49.601459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 200/397] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:49.606736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.607026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.607300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.632065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:49.633247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:49.663096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82621901-41b8-45e7-876c-3e8726718878 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:49.749091 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 198/398] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:49.787995 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82621901-41b8-45e7-876c-3e8726718878 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:49.788646 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:49.829965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:49.830172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:49.830172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.830172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.830172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.838389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:49.844398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:49.845376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:49.846194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 201/399] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:49.851595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:49.874406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:49.949740 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99d2f-4df6-4099-a7e0-fb44eeab62ff None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:49.957746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21f8e-65fe-4be0-a4a3-823eec8dac82 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:49.995989 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:49.996629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:49.996981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:49.997261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:49.998107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:49.998972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.000482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.000969 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.002217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.002694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.003181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.010937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 202/400] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:50.016420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.016901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:50.022134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:50.023033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:50.023838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:50.026984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 199/401] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:50.033240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.085723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:50.124288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c31cfe-e0a3-4783-93df-1dc470b2d4fc None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.158893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.159541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.159824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.160100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.161243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.162705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.163535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:50.164614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 203/402] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:50.170911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.200517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d327440-7dff-47b5-8b35-81d4c2f084b8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.277820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d528f6-576a-4284-8820-bd7f128b053e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.302714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.303369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.303656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.303934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.304670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:50.304977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.306461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.314244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.315234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.315532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.315813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.316998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.318569 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.326292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 204/403] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:50.327727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.332110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:50.340207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 200/404] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:50.350991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.437318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce259c-63d4-40c8-ab48-28439013ca38 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.473643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.474375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.475060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.475367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.477724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, group_id=cabcedf045dd4a8e9c7455c5e756dfc6, role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.479368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0944fe-ccee-44c2-9790-063a873a543d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.501848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 205/405] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 06:32:50 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/150d91154f9544c7a914d19f74d59fe8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:32:50.617705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617051f0-0f78-4831-a549-f6421b16733c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.666225 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.666897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.667182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.667463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.674972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.676034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb8ac-a3b8-41e7-a35b-c5bbf5c23832 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.676321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb8ac-a3b8-41e7-a35b-c5bbf5c23832 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.676478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea2f0-8c80-4934-a7f5-53b05ffb3375 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.676629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb8ac-a3b8-41e7-a35b-c5bbf5c23832 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.678450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 201/406] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:50.682565 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 206/407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:50.683944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.684234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.684510 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.689862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cb8ac-a3b8-41e7-a35b-c5bbf5c23832 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.715196 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:50.716568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:50.727316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:50.789446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:50.804443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 202/408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:50.830905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:50.861046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cb8ac-a3b8-41e7-a35b-c5bbf5c23832 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.972777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6299b2-eaeb-4c26-8a91-ec328037b639 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:50.978671 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:50.979349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:50.979637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:50.979918 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:50.981145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:50.982617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:50.989519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 207/409] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:50.990632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.011672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.012341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.012660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.012936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.025895 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:51.031890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:51.032780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:51.033568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 203/410] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:51.038954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.042592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.150463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95e9d9-3c52-46a6-a937-7d58df69f7ae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.182271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50584d10-3e41-4a37-b909-39e14940640c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.196282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.196911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.197198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.197478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.198237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:51.198548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.199930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.209106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 204/411] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:51.216988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.225809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.239579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.240273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.240560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.240846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.242855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6cd3fc4f946f0a18b22115976a1d7, group_id=cabcedf045dd4a8e9c7455c5e756dfc6, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.244300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b853a-07be-41fb-987a-afb3a3d4a53b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.268176 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 208/412] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 06:32:50 2022] PUT /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7/groups/cabcedf045dd4a8e9c7455c5e756dfc6/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:32:51.269138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1d6-9b22-4a88-af32-0cd666372b75 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.269435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1d6-9b22-4a88-af32-0cd666372b75 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.269720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1d6-9b22-4a88-af32-0cd666372b75 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.272789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 209/413] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:51.277637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.277858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.278174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.295638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.323370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:51.324673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:51.339026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116d9f-d31b-45e1-8274-9c447d7f63f5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.412106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.413032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.417913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.418433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] PATH_INFO: `/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.422318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] RBAC: Authorizing `identity:get_group(group_id=4e7b4cb9038e4d15934a9b37637f58d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.424027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.425880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 205/414] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:51.431609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.462234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 210/415] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:51.489134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.551544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda45a7-4c7a-4160-95ff-720b36871f63 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.583389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.598626 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.599517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.599937 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.600527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] PATH_INFO: `/v3/projects/0d277a3b91684c12b74997019b6d7be2` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.601901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] RBAC: Authorizing `identity:get_project(project_id=0d277a3b91684c12b74997019b6d7be2)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.603641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.605745 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 206/416] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/projects/0d277a3b91684c12b74997019b6d7be2 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:51.608819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92424b-aba0-4b81-a416-d3098470e35e None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.612826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.638316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.648758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.649405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.652284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.652284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.669549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:51.675593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:51.676495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:51.677320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:51.686023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 211/417] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:51.686922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.801119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f2c6e-5751-45c4-b34b-4d1c0b046054 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.812249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bbac3-322c-4521-8b98-8b066deedf95 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.848887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.849873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.850410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.850982 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.851895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] RBAC: Adding query filter params (name=member) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:51.852398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.854125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.869050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 212/418] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:51.871184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:51.879898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:51.880592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.880875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.881157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] PATH_INFO: `/v3/projects/0d277a3b91684c12b74997019b6d7be2/groups/4e7b4cb9038e4d15934a9b37637f58d7/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.883437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] RBAC: Authorizing `identity:create_grant(project_id=0d277a3b91684c12b74997019b6d7be2, group_id=4e7b4cb9038e4d15934a9b37637f58d7, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:51.884985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ef878-29a0-4d54-8635-91a0612636eb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:51.899670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 207/419] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jul 27 06:32:51 2022] PUT /identity/v3/projects/0d277a3b91684c12b74997019b6d7be2/groups/4e7b4cb9038e4d15934a9b37637f58d7/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:32:51.901353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea70d0-8d34-4072-bf9d-6602a636533b None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.901643 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea70d0-8d34-4072-bf9d-6602a636533b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.901979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea70d0-8d34-4072-bf9d-6602a636533b None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.903408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 208/420] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:51.909597 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:51.910430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:51.910973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:51.930362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:51.933592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:51.935101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:51.993885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb169-1556-484f-ba5a-1580426a0552 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:51.998031 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.021144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 209/421] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:52.036781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.037667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.038136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.038739 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.040023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.118308 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.120050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.150213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708953fe-8c60-43b8-b47c-1b74e0d257a1 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.178360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.188884 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.189685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.190101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.190474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.204459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:52.211220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:52.212278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:52.213369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 210/422] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:52.219475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.260493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 213/423] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:52.264066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.291345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.381307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69dfe-9afc-480b-aa62-dcbc1c1860c0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.398211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b2585-0815-4574-a6da-0b0ce16e4c7f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.417167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.418166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.418727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.419211 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.421099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.422833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.429183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 214/424] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:52.430656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.446262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.464182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.464805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.465084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.465361 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.466101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:52.466407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.468161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.485003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 211/425] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:52.495015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.602345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e3757-7086-46b1-be2c-0a76485ba8f1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.664786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.665354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.665632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.667371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.669233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.671313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.674419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 215/426] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:52.680230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:52.687611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7db90-70eb-465c-8d46-85e47b7b57fe None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.712036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.750872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.751561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.751847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.752126 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.790279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:52.821366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d82dc-1806-4dc5-ac8d-48f76166678d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:52.843808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.845320 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.884022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:52.884686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:52.884940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:52.889963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:52.889963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:52.889963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:52.889963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:52.901315 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 216/427] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=4372496541ff48858e12197c1d6b1e71 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:52.954596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 212/428] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:32:52.959889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.106843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58508f06-1f6a-46a3-8fbd-cfc2022cf9f4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.143160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.144177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.144652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.145147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] PATH_INFO: `/v3/projects/a3dd68a4366748e6a6e25223e07c10a6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.146529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] RBAC: Authorizing `identity:get_project(project_id=a3dd68a4366748e6a6e25223e07c10a6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:53.148161 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29545ec-5448-4eca-8774-50bded8c7721 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:53.150359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 217/429] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:52 2022] GET /identity/v3/projects/a3dd68a4366748e6a6e25223e07c10a6 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:53.155782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.172698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:53.236411 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e81dee-d3be-4689-9555-fd64dd976a62 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.236706 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e81dee-d3be-4689-9555-fd64dd976a62 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.236903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e81dee-d3be-4689-9555-fd64dd976a62 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.238400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 218/430] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:53.244500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.245051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.245528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.266637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:53.269294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:53.272750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:53.277135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688aa0b-8106-4ab0-90e1-61dcabacd991 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.316288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.317165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.317294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.317486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.320485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:53.322897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:53.326784 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 213/431] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:53 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:53.338966 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.352436 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 219/432] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:53.384749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.457576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:53.487037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eee7f-e67e-4b48-bcee-95bb1675bc4a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.528365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.529092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.529906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.530248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.531133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=a3dd68a4366748e6a6e25223e07c10a6) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:53.531452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:53.532887 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:53.533973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa513-07ca-4524-9b12-f195ae7009d3 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.540771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 214/433] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:53 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=a3dd68a4366748e6a6e25223e07c10a6&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:53.579792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.580480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.580763 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.581042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.593334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:53.596914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:53.604718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:53.606943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:53.609384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 220/434] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:32:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:53.614616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.727881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64c74-9f83-4f63-9d56-7b85c0805958 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.763837 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.764535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.764818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.765092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.765820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:53.766169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:53.767524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6cf6d-4bdb-4e41-9a6f-8d2e82455e5e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:53.776485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 215/435] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:32:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:53.782655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:53.861617 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddece-9996-4077-8b98-55daa2fe0832 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.861931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddece-9996-4077-8b98-55daa2fe0832 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.862222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddece-9996-4077-8b98-55daa2fe0832 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.863857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 216/436] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:32:53.869138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.869429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.869714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.882884 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:53.890274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:53.891500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:53.907843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7aa0bb-565d-4500-a809-742c7326831a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:53.957367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:53.957680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:53.957979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:53.958268 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:53.993965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 217/437] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:54.016269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.053653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.055986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.057719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.136271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe05eb-bbb8-452a-b656-0fbbde622cef None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.167683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 221/438] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:32:54.173371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.174328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.174989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.175283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.175547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.178379 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.188335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:54.194846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:54.195724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:54.201841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 218/439] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:32:54.202959 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.274124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.277575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132c73a-5120-47ec-8d5d-094e8b985b31 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.313562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.314240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.314526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.314804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.315971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.318005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.319594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d11f60-6905-468e-bb8a-f534abe165c7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.324121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 222/440] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:54.325011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.358577 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.359211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.359492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.359767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.360490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] RBAC: Adding query filter params (name=member) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:54.360801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.362461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2583b-9850-4a73-a1c0-9bc5b5c474ba None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.380002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 219/441] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:54.381229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e33f7-4b1f-49bd-8d4e-299c9c995b88 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.381513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e33f7-4b1f-49bd-8d4e-299c9c995b88 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.381787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e33f7-4b1f-49bd-8d4e-299c9c995b88 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.383155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 220/442] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:54.385471 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e33f7-4b1f-49bd-8d4e-299c9c995b88 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.424099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee80ad8-6e62-416b-acf5-445e53beb3bd None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.444490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.457530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.458201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.458479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.458751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.459887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.461320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eebce-01d9-4f79-86b5-b91f8814e3a3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.462934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 223/443] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:54.464019 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.464298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.464571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.482313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:54.483531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:54.490533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e33f7-4b1f-49bd-8d4e-299c9c995b88 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.512460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.524319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.524874 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.525154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.525462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.550260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 224/444] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:54.554382 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.598398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.599850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.647475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68923b7-a496-4dd6-95a9-d4887afecc1c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.655915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.684145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.684770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.685055 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.685330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.686869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:54.687186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.688615 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.694735 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 225/445] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:54.695505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.699553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 221/446] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:54.705484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.739379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.816498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17859baa-6116-4b7b-83fa-918e1bf9d635 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.822136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5cc7a-7ccb-4d44-8a37-a8646a706124 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.854241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.855049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.855336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.855641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.864899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:54.865535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:54.865864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:54.866169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] PATH_INFO: `/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:54.867327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] RBAC: Authorizing `identity:get_project(project_id=e8d6cd3fc4f946f0a18b22115976a1d7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:54.868753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:54.870740 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 222/447] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/projects/e8d6cd3fc4f946f0a18b22115976a1d7 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:54.871503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:32:54.876032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:32:54.877107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:54.877985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:54.878819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 226/448] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:54.886131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:54.909359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.976678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:54.981566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04c62-bc8a-4a01-9adc-6818192db3ca None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:54.987907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aaa2a-8b32-4649-b1bf-6a9d71c6ed39 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.015988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.016628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.016913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.017190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] PATH_INFO: `/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.018372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] RBAC: Authorizing `identity:get_group(group_id=cabcedf045dd4a8e9c7455c5e756dfc6)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.019807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.021002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.021393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 223/449] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/groups/cabcedf045dd4a8e9c7455c5e756dfc6 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:55.021616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.021899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.022206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.022919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:55.023227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.024569 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.027068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:55.038122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 227/450] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:32:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:32:55.039124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:55.119954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:55.127055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ca185-f2c5-4a0e-982c-adaf2355dc1e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.135652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ccbc3-7a0c-4fe9-944a-032457eb97e6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.160589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.161200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.161482 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.161757 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.162576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] RBAC: Adding query filter params (group.id=cabcedf045dd4a8e9c7455c5e756dfc6, role.id=4372496541ff48858e12197c1d6b1e71, scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:55.162889 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.164325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.169214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.170661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.170777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 224/451] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:55 2022] GET /identity/v3/role_assignments?group.id=cabcedf045dd4a8e9c7455c5e756dfc6&scope.project.id=e8d6cd3fc4f946f0a18b22115976a1d7&role.id=4372496541ff48858e12197c1d6b1e71 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:55.170868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.171048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.195189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:55.247451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.249137 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.351838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 228/452] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:32:55.357511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:55.459467 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b24a-8fec-452a-aebd-f87cc61d1277 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.497626 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.498962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.499246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.499591 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] PATH_INFO: `/v3/projects/0d277a3b91684c12b74997019b6d7be2` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.500818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] RBAC: Authorizing `identity:get_project(project_id=0d277a3b91684c12b74997019b6d7be2)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.502310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.504256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 225/453] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:32:55 2022] GET /identity/v3/projects/0d277a3b91684c12b74997019b6d7be2 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:55.510735 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:55.566722 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:55.613273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303dc58-15ad-42c5-8612-bbc789738b80 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.646618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.647258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.647538 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.647809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] PATH_INFO: `/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.648947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] RBAC: Authorizing `identity:get_group(group_id=4e7b4cb9038e4d15934a9b37637f58d7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.650399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb917996-dca5-4010-936b-06ee1ed3ce14 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.652289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 229/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jul 27 06:32:55 2022] GET /identity/v3/groups/4e7b4cb9038e4d15934a9b37637f58d7 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:55.658772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:55.687049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:55.758136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9d3a-0738-4725-8fd2-d2d5b2e393d4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:55.790547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:55.791156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:55.791439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:55.791715 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:55.792498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] RBAC: Adding query filter params (group.id=4e7b4cb9038e4d15934a9b37637f58d7, role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=0d277a3b91684c12b74997019b6d7be2) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:55.792809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:55.794269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1a878-d086-42bd-8087-cbaa52063b2c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:55.800116 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 226/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jul 27 06:32:55 2022] GET /identity/v3/role_assignments?group.id=4e7b4cb9038e4d15934a9b37637f58d7&scope.project.id=0d277a3b91684c12b74997019b6d7be2&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:58.238737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf26880-7cad-4282-ac43-70a6075b1393 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.239781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf26880-7cad-4282-ac43-70a6075b1393 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.240272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf26880-7cad-4282-ac43-70a6075b1393 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.242132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 230/456] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:58.249792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.251247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.252063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.275505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:58.278093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:58.366044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 227/457] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:32:58.393785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf26880-7cad-4282-ac43-70a6075b1393 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:58.501299 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf26880-7cad-4282-ac43-70a6075b1393 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:58.533991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:58.545252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:58.546164 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.546646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.547181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.557580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:58.559052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c7ce7178-b2b2-4ca1-8c12-0b41a512ccc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:32:58.585325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:58.586337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:58.587185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:32:58.587185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:32:58.587185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:58.587185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:32:58.587832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 231/458] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:58.593537 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:58.626765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:58.657504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822bfc-8256-4a37-be8e-843afe42a211 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.658154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822bfc-8256-4a37-be8e-843afe42a211 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.658657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822bfc-8256-4a37-be8e-843afe42a211 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.660466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 232/459] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:58.671196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def500c-aa64-4ecb-8b4b-90ffe1fce5b2 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.671482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def500c-aa64-4ecb-8b4b-90ffe1fce5b2 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.672132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def500c-aa64-4ecb-8b4b-90ffe1fce5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.697332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-0def500c-aa64-4ecb-8b4b-90ffe1fce5b2 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:58.698565 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-0def500c-aa64-4ecb-8b4b-90ffe1fce5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:58.782903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff195b-8f65-4615-8a2d-d8d933cf6003 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:58.828699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 233/460] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:58.836047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:58.836738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.837027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.837305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.838031 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:58.838427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:58.839805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:58.853109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e16414-9436-4c48-9cc7-86ba43f0be5c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.853409 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e16414-9436-4c48-9cc7-86ba43f0be5c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.853659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e16414-9436-4c48-9cc7-86ba43f0be5c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.854699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 228/461] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:58.855235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 234/462] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:58.855530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:58.861498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ee6e-9804-4699-9529-82ba4bbede50 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.861784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ee6e-9804-4699-9529-82ba4bbede50 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.862093 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ee6e-9804-4699-9529-82ba4bbede50 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.863489 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 235/463] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:58.866882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:58.867175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:58.867440 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:58.886702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:58.888355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:58.942935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:58.963681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20293605-68b1-49db-b29c-d69f8df7358c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:58.974041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 236/464] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:58.974041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.003066 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.004088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.004370 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.004646 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.007855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.016616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.018104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d45e7095-f8fc-482b-b367-fabaa63e94c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:32:59.033602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:59.034518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:59.035475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:32:59.035475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:32:59.035475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.035475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:59.035475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 229/465] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:59.036184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.036279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.036499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.055155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:59.056341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:59.083602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032edff-e478-4f2b-b1ef-cbfc81093798 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.118592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.119197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.119474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.119747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.120419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.121835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.130152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 230/466] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:32:59.130152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.157869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 237/467] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:58 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:59.157869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.205230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.252532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf63c1-43be-4db9-b424-d03cca73ef97 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.293172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.293915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.294066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18d9a-6f42-4b6e-a8c4-902a4236de79 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.294488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.295504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.298328 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.307854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.309732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:32:59.327531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:59.328452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:59.329333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:32:59.329333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:32:59.329333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.329333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:59.329333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 231/468] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:59.332133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.344800 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.345478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.346132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.347041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.347225 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:59.347835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.349516 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.374062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 238/469] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:59.378544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.456774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b36daf-8ab0-4340-a565-03ca410660da None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.475785 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.503286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.503938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.504223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.504498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.504626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350193d-0d7f-4203-b9fe-743eea73fc45 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.519059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.521266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:32:59.543039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:32:59.544002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:32:59.545324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:32:59.545324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:32:59.545324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:32:59.545324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:32:59.545682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.545682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.545682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.545682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.546472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 232/470] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:32:59.546472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.546768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:59.546768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.548025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.563773 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 239/471] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:59.565031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.576959 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.674468 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06714a-8e4a-4ad2-88b9-ac5da4735a8a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.679849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f960-9eec-48dd-90a5-228e822fc14d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.711301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.712167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.712445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.712720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.713426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:32:59.713728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.715162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.715281 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.715794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.716077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.716355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.717065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.720245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21336-6855-4ea7-890a-d41caa1e0750 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.727728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 240/472] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:32:59.728295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.744644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 233/473] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:59 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:59.745876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249dbf-7cf3-4a06-ba17-2dbd234d6437 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.746199 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249dbf-7cf3-4a06-ba17-2dbd234d6437 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.746527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f249dbf-7cf3-4a06-ba17-2dbd234d6437 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.748453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 234/474] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:32:59.749042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f249dbf-7cf3-4a06-ba17-2dbd234d6437 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:32:59.815613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.820658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:32:59.863556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dfdfb-ea5d-4095-aaf8-baef98fc163b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.903588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.904730 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.905217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.905672 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.906611 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.908486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b436a6-4bd7-4d9c-9328-b04235a2c883 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:32:59.930615 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f249dbf-7cf3-4a06-ba17-2dbd234d6437 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:32:59.938674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 241/475] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:59 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:32:59.940185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55179-646f-4900-8eff-63b76f563438 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.940694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55179-646f-4900-8eff-63b76f563438 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.941601 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55179-646f-4900-8eff-63b76f563438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.961446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-31d55179-646f-4900-8eff-63b76f563438 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:32:59.962771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-31d55179-646f-4900-8eff-63b76f563438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:32:59.983875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:32:59.984631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:32:59.984918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:32:59.985217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:32:59.985911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:32:59.987679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:00.017573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 235/476] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:32:59 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:00.057562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 242/477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:00.081925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:00.178598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:00.205341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36343b-412d-41a4-ad50-0221e7f453c2 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:00.242402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:00.243118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:00.243402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:00.243678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:00.244389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:00.245749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:00.253639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.sql.core [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 06:33:00.253639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 06:33:00.253639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [parameters: {'id': '0f601fac49ef416cbfb9d43d50a728e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 06:33:00.253639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86820) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:00.270054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:00.270977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application yield resource Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise self.value Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application session.commit() Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 06:33:00.271912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.session.flush() Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application compat.raise_( Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 06:33:00.272777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:00.273694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:00.274594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [parameters: {'id': '0f601fac49ef416cbfb9d43d50a728e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:00.275408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 06:33:00.276222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:00.277085 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 236/478] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 06:33:00 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:00.286040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d55179-646f-4900-8eff-63b76f563438 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:00.350102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:00.403365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d55179-646f-4900-8eff-63b76f563438 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:00.439742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:00.440395 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:00.440673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:00.440949 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:00.453317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c92aa443-2bf2-472f-9d51-000c1d959e36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:00.458586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:00.459602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:00.460413 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 243/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:00.468049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:00.605131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d45d8-65d7-4a22-866f-21af49ad33c3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:00.656125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:00.656756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:00.657035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:00.657306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:00.661001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:00.661001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:00.661001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6f996-4bfa-4998-9e74-cf9eccfeb426 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:00.673044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 237/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:01.406920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2ab-6f60-477a-9b4a-6f351a78bea4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:01.407434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2ab-6f60-477a-9b4a-6f351a78bea4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:01.407544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2ab-6f60-477a-9b4a-6f351a78bea4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:01.409059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 244/481] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:01.414196 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:01.414588 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:01.414771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:01.424594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:01.438899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:01.440084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:01.532221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 238/482] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:01.567330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc2ab-6f60-477a-9b4a-6f351a78bea4 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:01.687767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc2ab-6f60-477a-9b4a-6f351a78bea4 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:01.707053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:01.732504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:01.733137 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:01.733419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:01.733691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:01.749046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:01.754087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:01.755046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:01.755856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 245/483] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:01.762801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:01.922434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd365bd6-6c2b-453e-8d4a-489082f899f3 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:01.988259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:01.988902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:01.989186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:01.989466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:01.990263 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:01.990575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:01.991952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:01.993487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:02.006289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 239/484] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:02.011741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.176808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5804c90-3f91-4a84-962d-5e2c37a5e92b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.235595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.236497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.236785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.237062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.257583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:02.259086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-7eb57ef9-110c-489a-b165-c5703536d692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:02.276260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:02.277184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:02.278111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:02.278111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:02.278111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:02.278111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:02.278111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 246/485] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:02.284677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.380364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb45f0-51ea-4fbe-a19a-2a0b5d589588 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.380712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb45f0-51ea-4fbe-a19a-2a0b5d589588 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.380905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb45f0-51ea-4fbe-a19a-2a0b5d589588 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.382554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 247/486] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:02.387813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.388098 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.388909 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.410016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:02.411380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:02.417881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240db66-9457-4dbd-addf-c6bf1563cf06 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.434564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:02.467236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.467870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.468160 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.468426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.469132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:02.469442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:02.470844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:02.484076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 240/487] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:02.489632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.493611 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 248/488] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:02.551567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.590256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:02.661668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18063a23-717f-4027-872d-0ef20d40e049 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.699237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.699792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.700352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.700484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.707535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1ce7e-b021-4b26-8198-bfddb110afa5 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.708159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:02.709703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e846a05d-d0a4-43a5-ad6f-3419bce678a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:02.724413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:02.725525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:02.726457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 241/489] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:02.727029 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d3b4-8b62-4422-a4d0-73344c3662ba None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.727095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d3b4-8b62-4422-a4d0-73344c3662ba None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.727548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d3b4-8b62-4422-a4d0-73344c3662ba None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.728823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 242/490] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:02.734375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.734561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.734945 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.751351 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.752300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.752723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.753163 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.755381 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:02.756783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:02.766184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:02.773191 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:02.774295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:02.775312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:02.782405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 249/491] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:02.783145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.803288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:02.838828 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 243/492] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:02.839695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.891135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:02.892693 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9885a0-35f1-4bd2-9dc7-cc2b9c9907a3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.931376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.932608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.933087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.933561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.935001 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:02.936153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:02.937889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:02.947138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5fd7d-701c-4dce-9fe7-5993defd452c None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:02.974226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 250/493] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:02.975331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:02.988861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:02.989792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:02.990118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:02.990401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:02.991128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:02.991444 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:02.992841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4789396-bf0d-4315-bc74-20b7f2e8d96f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 244/494] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbcffb-ecff-431d-9ef4-88d00ab3134d None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbcffb-ecff-431d-9ef4-88d00ab3134d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbcffb-ecff-431d-9ef4-88d00ab3134d None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 245/495] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:03.006397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbcffb-ecff-431d-9ef4-88d00ab3134d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.059864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.089092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61663523-9236-4d32-b72f-1350dc429fb9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.118611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbcffb-ecff-431d-9ef4-88d00ab3134d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.127090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.127737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.128020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.128296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.129572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.140392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:03.146976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.147881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:03.148705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 251/496] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.149396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.161495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.162146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.162426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.162702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.232070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.233492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.255928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ee3bb-5f0c-4a11-a63e-aa2c4b4957c2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.290649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.291293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.291574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.291870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.293876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.305200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.307050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-71e99398-e9b0-4f70-9227-def665824d33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:03.321878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.322857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.323703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:03.323703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:03.323703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.323703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:03.323703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 252/497] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.324630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.324922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.325202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.343203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 246/498] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:03.343857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.346930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:03.348142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:03.355060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.420657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 253/499] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:03.422530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.451926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579a424-d3b6-430d-8ab7-0ccdcdb6743b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.487040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.487574 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.487853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.488128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.488850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:03.489148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.490558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.498544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 247/500] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:03.499326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.527702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755d7d-e9d8-4e4c-9474-10084cc16264 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.544004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.562359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.562986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.563272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.563547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.564235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:03.564540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.565916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcc21-dfe9-435a-a4b9-0d70e50abd56 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.567649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.577577 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 254/501] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:03.578819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff71681-c6ef-4261-aef3-28dcda7905f8 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.579106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff71681-c6ef-4261-aef3-28dcda7905f8 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.579388 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff71681-c6ef-4261-aef3-28dcda7905f8 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.580588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 255/502] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:03.584898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff71681-c6ef-4261-aef3-28dcda7905f8 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.604709 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f077-7cff-40e7-b2dd-6a56049335a7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.637415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.638094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.639245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.639245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.649782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.651283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:03.667131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.668079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.668939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:03.668939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:03.668939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.668939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:03.668939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 248/503] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.669207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.688755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff71681-c6ef-4261-aef3-28dcda7905f8 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.722653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.723292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.723571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.723850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.734204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:03.739819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.740718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:03.741620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 256/504] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.742568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.780360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.783055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce30c7-58b2-42c6-9233-0f151427248d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.804706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:03.816509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.817144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.817430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.817711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.828917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.830560 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-6eb75a6e-01a7-4a65-8e92-0d27d7f02335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:03.844633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.846554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:03.846554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:03.846554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:03.846554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:03.846780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f57624-8e7e-4e74-a3be-d85e83e1fc8a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:03.846862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 249/505] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.846931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.847166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.847440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.865202 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:03.866419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:03.879776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:03.880415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:03.880697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:03.880972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:03.887786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:03.889801 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:03.902362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:03.903438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 257/506] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:03.909682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:03.933552 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 250/507] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:03.934580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.002216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.008096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eab20-8a60-469a-b461-038d622d85f6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.033900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd248fb2-ac57-45ed-98c9-0ef7adef5f91 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.036266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.042275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.042888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.043168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.043450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.044131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.045037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.046592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.058273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 258/508] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.059074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.069226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.070201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.070201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.070454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.071127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.071436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.072793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.080694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 251/509] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.081466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.161703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78da00b-0476-40f0-8e64-8b785fe3c1d7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.179291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8ddf0-424a-4b74-a460-c93be0fdb4e1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.196016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.196635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.196915 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.197193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.197877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.198216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.199570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.211182 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 259/510] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.211810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.216150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.217577 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.217811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.218126 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.218931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.219246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.220985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.229591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.258438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 252/511] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.259200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.283837 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.311813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ccd9c-d0fc-4725-9009-8a0d35343aac None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.344618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.345254 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.345540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.345815 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.355999 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.357488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.358419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3645b-aab2-49e1-8038-24f7fbe8bb4d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:04.372029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:04.373112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 260/512] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:04.374188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.396100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.396736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.397013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.397288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.403815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.405836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:04.418812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:04.420274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 253/513] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:04.421231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.446133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.482456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74e83a-7bf5-464b-9184-f120bb62df73 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.505952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.517202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.517781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.518092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.518367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.522158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e199818-e8de-49db-81e9-c58cc0025ab6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.528547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.530495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:04.545128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:04.546048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:04.546920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:04.546920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:04.546920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:04.546920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:04.546920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 261/514] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:04.547185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.559390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.560089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.560885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.561165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.568333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.569781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:04.581374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:04.582294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:04.583210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:04.586290 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 254/515] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:04.587274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.644865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9475c-03de-46a2-b93e-9692248f44b1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.668197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.682455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.683070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.683350 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.683624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.686381 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bac9b-173c-42b2-81b1-16e703cb2202 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.718560 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.719162 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.719443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.719720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.720405 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.720708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.722103 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.734181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 255/516] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.734963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.740044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.756127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.757580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.848317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4be8-962b-4b4a-8e1c-89704ba95581 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.854270 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 262/517] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:04.855098 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.877429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.883405 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.884013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.884295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.884570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.885308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.885611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.887082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:04.905188 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 256/518] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:04.906026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:04.954026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e541562-fb8f-4772-af2e-c31cba52ba82 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:04.967722 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:04.987770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:04.988376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:04.988654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:04.988928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:04.989608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:04.989908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:04.991287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.002136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 263/519] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:05.002888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.010290 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ccd7-cb61-4808-85cd-e5a381762663 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.044261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.045180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.045463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.045738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.046842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:05.047155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.048613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.065616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 257/520] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 06:33:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:05.066435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.085508 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.103908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de01f89-4fd6-422a-92d6-c2f8e35e7b02 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.136282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.136888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.137165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.137437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.138823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.140252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.163461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 264/521] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:33:05 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:05.164418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.164515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4489a-c255-4efb-b1f9-244910c5122d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.215465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.231362 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.232307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.232682 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.233061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.245425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.247326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:05.265534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:05.266691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:05.267780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:05.267780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:05.267780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:05.267780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:05.268731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 258/522] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:05.274377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.274885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0e25b-5d70-4855-9df3-119d0e427c80 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.312429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.313056 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.313337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.313612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.314450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:05.314764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.316208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.319208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.323511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 265/523] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:05.324901 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.417334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fd75-93be-4ed7-945f-7640621227a4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.458096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e70dc-2cd5-4777-ab42-cbbf1d883077 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.462742 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.463413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.463694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.463971 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.475178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.478524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:05.494807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:05.495949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:05.497053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 259/524] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:05.497985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.505244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.505874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.506349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.506684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.544262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.583557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.587432 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.589281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.643605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45955da-c233-46e5-a372-e2435aef8007 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.679669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.680456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.680883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.681264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.682330 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:05.682559 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.684088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.697052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 260/525] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:05.698131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.706968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 266/526] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:05.708003 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.798706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.807124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360cd9f-a2be-4dac-b9b8-9a24b9858cd5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.823391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:05.827419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0e2a5-e961-44cb-a8dc-80600c0df8b3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:05.844727 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.845651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.846213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.846627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.847660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:05.848107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.849779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.864280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:05.864931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:05.865220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:05.865491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:05.870888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 261/527] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:05.871991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.877731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:05.879224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:05.893201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:05.894290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:05.895295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:05.898372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 267/528] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:05.899271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:05.983570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ad0e0-3a40-4e0a-8f0e-28f298f6ecea None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.007981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f1c2d-195d-4938-931b-003aada7392d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.021284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.022147 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.022662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.023077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.035070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.036952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.045285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.045672 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.045985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.046255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:06.054404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:06.055496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:06.056437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:06.056437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:06.056437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:06.056437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:06.058643 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 262/529] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:06.059565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.063701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.082547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.126017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.127369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.170425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1fbd2-b402-42c3-9371-8daf71b8f5d6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.205901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.206801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.207222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.207660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.208645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.209102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.210886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.224557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 263/530] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:06.225533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.242480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 268/531] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:06.243334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.292353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.305294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.336791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078bf13-91e2-4d42-b295-f5b6743f86a1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.347757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c79603-f4b1-4a6f-9003-7bb919294baf None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.378387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.379287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.379706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.380129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.380822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.381129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.382514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.384099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.384911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.385341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.385766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.386722 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=665904332c794638bd3a890fe602cfb8) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.387184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.388841 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.400022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 264/532] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:06.400022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.400646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 269/533] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/role_assignments?user.id=665904332c794638bd3a890fe602cfb8&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:06.401688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.518948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659a33-5c71-4ac7-b3b9-df83c465f9f6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.542718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d758c-1e4f-4469-a337-e9dd5c413586 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.547009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.561810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.562520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.562799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.563072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.571224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.577074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.578880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:06.594721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:06.595669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:06.596955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:06.596955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:06.596955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:06.596955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:06.596955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 265/534] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:06.598966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.599897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.600446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.600922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.605983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.614770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.621984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:06.645546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:06.646561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:06.647476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:06.653806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 270/535] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:06.655339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.717453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5e6a9-bcd4-4f9e-bef6-b42d1a6aaf0c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.762518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.766466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.766755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.767037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.767739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.768046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.768386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fbfec-3b9e-44e8-9c37-64ab11b09e26 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.769579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.796076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 266/536] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:06.801707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.810239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.811148 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.811597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.811979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.812844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.813264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.814830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.833588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 271/537] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:06.839805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:06.866382 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.890269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:06.945717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc1182-f20e-405c-8f94-b7fbae2bab5d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.981043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e301f4-e381-4b10-abbb-1b32712a8306 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:06.982757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:06.983513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:06.983905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:06.984908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:06.985179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=cb171d91fc464ac1ae2ade235f5c7608) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:06.985596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:06.987433 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:06.998195 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 272/538] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/role_assignments?user.id=cb171d91fc464ac1ae2ade235f5c7608&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:07.021249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:07.022211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.022637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.023057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.035280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:07.037001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:07.064060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:07.065121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:07.066152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:07.068302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 267/539] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:07.073826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:07.209064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d95f6-7243-4206-872e-29f888d1ac35 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:07.219647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:07.246197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:07.246819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.247096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.247898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.248649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:07.248955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:07.250423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0046f22-17bf-46c9-bc99-9db76135fb5d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:07.263121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 273/540] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:07.269041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:07.373644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:07.379744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5f40c-cfc1-428a-929e-d074382209ae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:07.414694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:07.415313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.415593 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.415869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.416656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b9f7fa9ae17a4ed7a717ed044eabca83) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:07.416965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:07.418431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230332f-229b-48d3-b16b-e3adb21dbff4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:07.432181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 268/541] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/role_assignments?user.id=b9f7fa9ae17a4ed7a717ed044eabca83&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:07.537246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f173a1-ce3c-4446-9166-61b649416dc2 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.537580 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f173a1-ce3c-4446-9166-61b649416dc2 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.537833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f173a1-ce3c-4446-9166-61b649416dc2 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.539502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 274/542] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:07.546047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.546361 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.546620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.554036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7717c-66f6-4e39-8360-2fcdf15aa049 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.555044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7717c-66f6-4e39-8360-2fcdf15aa049 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.555530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7717c-66f6-4e39-8360-2fcdf15aa049 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.557757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 275/543] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:07.566243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00d809-192f-45e4-9056-addc642603ab None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.566862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00d809-192f-45e4-9056-addc642603ab None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.567365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00d809-192f-45e4-9056-addc642603ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.574797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:07.577313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:07.590063 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-2a00d809-192f-45e4-9056-addc642603ab None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:07.591734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-2a00d809-192f-45e4-9056-addc642603ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:07.681972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 276/544] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:07.697292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 269/545] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:07.704515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00d809-192f-45e4-9056-addc642603ab None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:07.720425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:07.840687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:07.841970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00d809-192f-45e4-9056-addc642603ab None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:07.850289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:07.853513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463def9-5aed-4613-befe-60ea00951b62 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:07.890925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:07.891953 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.892470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.892948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.904636 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:07.910106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:07.910985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:07.911782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:07.912393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:07.912393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:07.912393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:07.912393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:07.915934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 270/546] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:07.921475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:07.942073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:07.947237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:07.948198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:07.950076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 277/547] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:07.958376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.039750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8f44f-856f-441a-b22a-71d2ea1b9694 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.078856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.079871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.080355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.080928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.081841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:08.082371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.084195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.093678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 271/548] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:08.099148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.128002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:08.186825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b723631-667f-4972-a98d-55337bb4de1e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.221858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.223271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.223271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.223271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.224164 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:08.224581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.226102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.235396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 278/549] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:08.242963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.289331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8051e-22a3-4b39-b9ef-2355968696bc None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.292249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:08.359602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.360566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.362271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.362695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.381954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.383473 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:08.408660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:08.410912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:08.411951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:08.411951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:08.411951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:08.411951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:08.411951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 272/550] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:08.418845 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.424472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d91eb-586c-4fd9-a9de-5fe4e3d1d0ae None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.468385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.469384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.469960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.470459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.484103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.486263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:08.503917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:08.504857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:08.505741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:08.505741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:08.505741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:08.505741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:08.508246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 279/551] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:08.517912 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.557001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:08.601604 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732924-1266-41ab-9d2b-4c500ee80c5c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.631732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:08.642827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.643847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.644346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.644857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.645850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:08.646411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.648394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.662659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 273/552] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:08.670046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.681377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f17212-fd91-43d2-979b-ce3ce67f62ec None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.738885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.739505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.739788 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.740073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.740927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:08.741236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.742994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.773860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 280/553] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:08.789383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:08.829424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0749-b4a5-46a8-8327-d4939d90420d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:08.860073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:08.879169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:08.880412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:08.880865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:08.881324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:08.894087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:08.896592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:08.915641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:08.916671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:08.917596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:08.921216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 274/554] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:08.928237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.080935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552b77-a918-468c-90ca-ce4be914709c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.116029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a941-8dcc-4500-a45b-8bac966fd879 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.155247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.155877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.156161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.156439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.157243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:09.157555 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.159238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743d16-2844-4e60-b1ee-ec46e57f4e58 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.178194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.186820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.188012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.188498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.188998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.197755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.200319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.210001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 275/555] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:09.215459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4d2d-823d-47ee-a4a5-024eb7901e8e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.216474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4d2d-823d-47ee-a4a5-024eb7901e8e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.216631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4d2d-823d-47ee-a4a5-024eb7901e8e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-dc1eeb22-0ed3-44a0-816c-04279aae8ec4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:09.219791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:09.221106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:09.223659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:09.224748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 276/556] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:09.225479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4d2d-823d-47ee-a4a5-024eb7901e8e None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.227144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 281/557] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:09.229047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.230590 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.231167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.254981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:09.256649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:09.340514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.396412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 282/558] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:09.401067 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.419570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4d2d-823d-47ee-a4a5-024eb7901e8e None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.466743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.467579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.467964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.468733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.530174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd29f-e48f-4e6e-b459-87fef8c17bb5 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.553589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.555392 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.559428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.573782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.574581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.575067 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.576041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.577049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:09.577462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.579025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.603293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 283/559] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:09.603293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.608550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.675539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 277/560] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:09.676641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.713975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43599-290d-4f04-86ca-a16b13f9968e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.751767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.752501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.752869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.753211 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.764753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:09.770083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:09.771031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:09.771883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:09.776002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 284/561] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:09.776955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.789269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a7468-e25a-4db6-b40b-66d62de859e6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.825037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.827196 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.828110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.828393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.828662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.839607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.841109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.843109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-6ae9a267-b238-4d98-ae91-dffa1fad3c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:09.859710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:09.861040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:09.862262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:09.862262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:09.862262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:09.862262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:09.865869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 278/562] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:09.867210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8e7e2-8e24-418b-8d1a-8f6b3ab92bd8 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.867492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8e7e2-8e24-418b-8d1a-8f6b3ab92bd8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.867767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8e7e2-8e24-418b-8d1a-8f6b3ab92bd8 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.870053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 279/563] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:09.871306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8e7e2-8e24-418b-8d1a-8f6b3ab92bd8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.889821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4993f-e897-41aa-8abc-795c55f3b895 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:09.926229 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:09.927121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:09.927512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:09.927895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:09.939510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:09.941041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:09.955317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:09.956361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 285/564] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:09.957456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:09.977860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8e7e2-8e24-418b-8d1a-8f6b3ab92bd8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.012034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.012647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.012971 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.013235 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.013971 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.014278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.015637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5897-49a0-4343-8690-56cb4c1465a7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.023428 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 280/565] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.024562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd824f4-d42e-4601-a7b7-9c148193fa45 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.024852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd824f4-d42e-4601-a7b7-9c148193fa45 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.025133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd824f4-d42e-4601-a7b7-9c148193fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.043050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6cd824f4-d42e-4601-a7b7-9c148193fa45 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:10.044322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6cd824f4-d42e-4601-a7b7-9c148193fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:10.053557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.063575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88bda8-b49d-4718-955e-ee0c826bc086 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.094188 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.097910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.098564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.098871 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.099150 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.099836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.100146 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.101495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.110526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 281/566] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:10.112087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345465ed-0ea8-4e83-be6f-c37c90665e51 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.112364 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345465ed-0ea8-4e83-be6f-c37c90665e51 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.112639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345465ed-0ea8-4e83-be6f-c37c90665e51 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.118094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 286/567] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.118221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 282/568] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:10.118320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.118397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345465ed-0ea8-4e83-be6f-c37c90665e51 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.217338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c4d1-e98a-4c5c-bee9-58741c2b6b77 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.225452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345465ed-0ea8-4e83-be6f-c37c90665e51 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.250648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.251264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.251550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.251827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.252518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.252826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.254210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.259557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.260191 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.260480 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.260764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.266027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 287/569] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.266819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.271869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.273287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.282381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-02de5709-ce90-41c9-8259-f164341d09ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:10.288741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:10.289789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:10.291024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:10.291024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:10.291024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:10.291024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:10.291024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 283/570] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:10.291359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.291498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.291778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.309015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:10.310819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:10.341083 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.371475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39453d4-ea08-429a-ad64-d3e6dcb2e6a2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.380189 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 284/571] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:10.380847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.405083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.405718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.406028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.406304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.412776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.414233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:10.425760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:10.426876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:10.427792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 288/572] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:10.428280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.490288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652e441-7135-4b36-bd4e-d603fc9d8619 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.500810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.525901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.526556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.526835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.527114 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.530902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65277c33-9399-42e2-843d-24649e772dc6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.537050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:10.541892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:10.543025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:10.544566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 285/573] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:10.545428 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.569575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.570329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.570674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.571011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.579393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.580660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.580993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:10.592652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:10.593764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 289/574] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:10.594769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.654962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d729238-7db1-4fac-b2b6-fcbe95d26f42 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.688641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.689261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.689550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.689832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.690557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.690866 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.692260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.695597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232fe8-3b8d-4dee-8425-4e491c553659 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.703329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 286/575] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.704127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.724453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.729686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.730359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.730636 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.730907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.740854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:10.746152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:10.747850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 290/576] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:10.748707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.793438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.805892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e979-8444-4306-8d47-0589ad12e443 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.839069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.839676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.839954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.840226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.841531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.841991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.843410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.847151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fdcec-cf80-4ca8-b744-3b0f3ba20e66 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.854849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 287/577] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.855626 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.880823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.881523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.881878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.882249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.883032 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.883416 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.884767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:10.892643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 291/578] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:10.893423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:10.949592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:10.958633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8ac28-903c-44c8-b042-a3a52e9fe4d7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.992018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:10.992630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:10.992910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:10.993188 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:10.993928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:10.994250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a1ed9-c1e7-499a-a337-49484057253d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:10.994349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:10.995687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.007981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 288/579] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.008664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.022645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.029247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.029887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.030200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.030480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.037992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.039417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:11.050896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:11.052013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 292/580] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:11.057726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.113053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aef80-9199-4c8e-acdc-34547d35b0ae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.146517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.147130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.147414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.147695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.148406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:11.148715 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.150119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.156597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab48b-9981-4c0c-82bd-fc289e53ba6c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.157859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 289/581] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.158672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.172070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.192776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.193456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.193728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.194028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.208403 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=b426ebd709e942819f19926f2e3668d2, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.209779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.221656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 293/582] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:11 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:11.222480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.240730 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.279303 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30999-717b-4c4e-ac6f-1157ec169238 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.317233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.318081 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.318363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.318728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.334994 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.336481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.339476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f06a-bc23-4c00-a636-5b6ed109653e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:11.352606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:11.353595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:11.354605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:11.354605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:11.354605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.354605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:11.359487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 290/583] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:11.360452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.385798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.386479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.386758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.387035 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.387939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=4a1b7d1e6073474081c39f043ac1ba26) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:11.388316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.389750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.395741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 294/584] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/role_assignments?user.id=4a1b7d1e6073474081c39f043ac1ba26&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.396465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.440832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.479635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986754ee-4600-4dcf-afbb-31deb781b790 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.505796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.519226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.519865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.520558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.520853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.522126 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:11.522483 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.524279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.531929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833e502-9c47-4021-8481-09a89c18ffe8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.550215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 291/585] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.551117 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.575463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.576364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.576814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.577230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.588285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.590037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:11.605684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:11.606921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:11.607814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:11.607814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:11.607814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.607814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:11.614437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 295/586] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:11.615071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.673002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2579e-27a6-47a3-a39b-4aac081e364b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.710985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.725132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.726211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.726772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.727058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.728021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:11.728731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.730460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.735080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78850ad3-8c09-446d-ad2d-6a115d32c00a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.750119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 292/587] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.751009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.776503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.777151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.777430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.777702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.789599 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.791325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.800436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:11.809016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:11.810142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:11.811230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:11.811230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:11.811230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:11.811230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:11.817295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 296/588] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:11.818523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.865632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbde9a-7506-4123-a32f-fe1bb9fd32eb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.907589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.908912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.909749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.910230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.911321 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:11.911774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.914028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.932055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 293/589] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:11.933201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:11.933564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234698-0f70-470b-999b-eb68c2f9f4da None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:11.965908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:11.971830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:11.972613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:11.972984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:11.973331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:11.983022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:11.984472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:11.996941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:11.998727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:11.999870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:12.005492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 297/590] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:12.006550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.062596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.068021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af1fff-633b-4c4e-89d8-17404dc5825f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.112151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7aee-2b57-4bc3-996e-3e86824a8ab9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.112691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.113474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.114172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.115456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.116310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:12.116779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.118307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.138269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 294/591] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:12.139776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.157009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.158654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.159261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.159795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.171613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.173872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:12.189518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:12.190492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:12.191657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:12.194027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 298/592] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:12.195068 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.248199 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7b1d-1c45-422f-8ef3-f3241f3b08af None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.250997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.285708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.286372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.286659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.286939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.287737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:12.288144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.289604 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.303862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2b870-adca-45cd-88f7-7d5af15eb3df None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.312863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 295/593] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:12.313770 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.333294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.341321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.342461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.342996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.343483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.350830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.352554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:12.366221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:12.367332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:12.368272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:12.375355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 299/594] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:12.376525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.425163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fca2b-d453-4d58-bd56-1fe96af9a6a8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.461198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.461753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.462078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.462360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.463159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:12.463472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.464906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.483936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 296/595] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:12.484781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.485483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730669f-4e88-46bf-9732-b4f8cd05d494 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.500456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.523825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.524837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.525364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.525854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.537285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.539077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:12.554751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:12.555743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:12.556627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:12.556627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:12.556627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:12.556627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:12.559018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 300/596] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:12.561095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.586901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.619234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083717e3-d271-4013-9e37-c294e7e6c962 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.657830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.658505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.658791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.659070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.659970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:12.660236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.661665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.679827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 297/597] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:12.679827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.685236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868bd21-91f8-4d87-befd-ff7b76cffd30 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.720042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.721062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.721595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.722111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.734528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.736596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:12.752505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:12.753375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:12.754661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:12.754661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:12.754661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:12.754661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:12.756513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 301/598] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:12.757484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.776053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.787747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3a659-cd1d-4813-8e1f-5ee6785913bd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.825928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.826821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.827240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.827688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.828627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:12.829082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.830738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.833485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:12.851398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 298/599] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:12.852584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.861282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2615-f810-4ba4-8e4f-066874f32a84 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:12.896560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:12.897391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:12.897765 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:12.898563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/665904332c794638bd3a890fe602cfb8/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:12.913495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=665904332c794638bd3a890fe602cfb8, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:12.915118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:12.934029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 302/600] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:12 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/665904332c794638bd3a890fe602cfb8/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:12.936061 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:12.994644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926aa83-c786-4989-9ad3-6469d8a02b26 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.071234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.072127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.072560 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.073065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.074338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.076524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:13.079970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.082706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.108771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 299/601] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:13.117758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.142099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a685e4-a9e5-4f40-a38d-ff860b3eb7f1 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.191221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.192147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.192682 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.193377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.193666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.203099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.204758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:13.216900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:13.218053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:13.219207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:13.220437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 303/602] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:13.225757 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.248610 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6f05-da35-4baa-b1de-6b425d08f6cd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.305545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.306212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.306492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.306768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.318403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.319839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:13.342404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:13.343561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:13.344666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:13.347094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 300/603] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:13.352868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.375322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039364-17fe-42be-bef8-8477e294d0fa None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.407803 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.418554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.420306 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.420658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.420999 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.421852 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:13.422250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.424272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38759b0-650b-4e6b-899f-52355e429c9b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.441138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 304/604] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:13.447115 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e30577-1e09-4849-9fa0-57952992a2d1 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.447410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e30577-1e09-4849-9fa0-57952992a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.447692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e30577-1e09-4849-9fa0-57952992a2d1 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.449122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 305/605] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:13.449976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e30577-1e09-4849-9fa0-57952992a2d1 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.489336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a02bed-e04a-4fe1-b64e-108721892d9b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.507502 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.528025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.529576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.530125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.530706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.531786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:13.532299 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.533979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d1d8b-6393-41c7-8f56-1f1e4ea315e1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.548161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 301/606] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:13.549803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590be5-a340-4391-98fc-fec434757105 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.550330 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590be5-a340-4391-98fc-fec434757105 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.550809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590be5-a340-4391-98fc-fec434757105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.568279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e30577-1e09-4849-9fa0-57952992a2d1 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.569033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-25590be5-a340-4391-98fc-fec434757105 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:13.570251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-25590be5-a340-4391-98fc-fec434757105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:13.603042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.603628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.603950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.604227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/cb171d91fc464ac1ae2ade235f5c7608/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.621418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=cb171d91fc464ac1ae2ade235f5c7608, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.623454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217aa9a2-f6ab-42bf-be99-a069922272ea None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 306/607] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:13 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/cb171d91fc464ac1ae2ade235f5c7608/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570eac-901c-4127-99cd-f37b5e5e219c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570eac-901c-4127-99cd-f37b5e5e219c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8570eac-901c-4127-99cd-f37b5e5e219c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 307/608] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:13.645969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8570eac-901c-4127-99cd-f37b5e5e219c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.661441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 302/609] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:13.663165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.663685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.664003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.688487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.691804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:13.693186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:13.768754 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8570eac-901c-4127-99cd-f37b5e5e219c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.798020 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 303/610] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:13.799205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.824825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.825890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.826433 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.826905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b9f7fa9ae17a4ed7a717ed044eabca83/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.843562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=b9f7fa9ae17a4ed7a717ed044eabca83, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:13.845411 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:13.851816 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:13.862279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 308/611] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:13 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b9f7fa9ae17a4ed7a717ed044eabca83/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:13.863501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:13.936052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe8f2-ec54-4ba0-a50d-a698b1d4772d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:13.978345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:13.979018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:13.979619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:13.979905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:13.995190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:14.002584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:14.003822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:14.006478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 304/612] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:14.013991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.035341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.059042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7e1fb-3f34-45e4-a276-52e5a27e4e38 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.107578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.108305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.109204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.109541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.124945 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:14.131768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:14.132697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:14.133524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:14.134204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.134848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 309/613] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:14.141124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.145418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869df32-f3ba-4ee0-84bc-8681904bfb94 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.193144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.194285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.194580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.194861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.195886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:14.196198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.198103 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.208228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 305/614] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:14.213587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.323949 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abe7be-cd9f-4622-ad64-54e78fae6c20 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.363292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.363916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.364198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.364475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.365353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:14.365679 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.367197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.368280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4c2d-c8e3-4e9f-bc9d-3bababf5c9d9 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.378099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 310/615] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:14.384715 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.420334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.420982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.421266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.421545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.430301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.432989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.434652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:14.449638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:14.450695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:14.451651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:14.451651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:14.451651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:14.451651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:14.451651 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 306/616] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:14.456606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.469847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.551930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95cd4a-81d2-4f7b-999a-d8182f6f54a7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.585662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.586336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.586614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.586888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.599374 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.601303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.613360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2c7c-bf32-49e6-b706-0cf6d3039f2c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:14.618250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:14.619145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:14.620059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:14.620059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:14.620059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:14.620059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:14.620059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 311/617] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:14.625430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.691600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.702978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.703664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.703921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.704201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.704905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:14.705214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.706624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.720070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 307/618] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:14.725305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.772079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10645-0e28-46b3-835f-495972e5fa78 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.814723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:14.848618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1a52f-a0ab-4b9b-a6a5-3f6c21f52758 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:14.860613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.861503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.861778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.862077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.862768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:14.863071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.864441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.884940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 312/619] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:14.891329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:14.896242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:14.896894 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:14.897173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:14.897448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:14.911870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:14.913311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:14.936110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:14.937772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:14.938793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 308/620] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:14.946952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.089603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319a3c-6231-4649-93e5-e5c1a80630aa None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.095848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc837f-31ca-4705-b0a4-046e2ec5a6ca None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.122851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.128638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.129461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.129905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.130657 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.131457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:15.131764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.133240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.152779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.154851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 309/621] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:15.160111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.190215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.190880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.191164 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.191434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.201285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.202745 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:15.217354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:15.218380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:15.219334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 313/622] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:15.226014 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.348681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189190d-68ee-48ff-ab2a-268523618f6d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.389613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.390280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.390565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.390858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523b04-052c-4bdd-9870-b1f006d5ad35 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.391034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.431602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.432222 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.432500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.432771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.433998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:15.434316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.436617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb702b44-fc04-4b43-8b1c-90be1febf694 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.453108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.458112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 314/623] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:15.459564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad281b2-79aa-4b68-8547-c2e18d977afd None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.459855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad281b2-79aa-4b68-8547-c2e18d977afd None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.460128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad281b2-79aa-4b68-8547-c2e18d977afd None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.465399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 315/624] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:15.465399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad281b2-79aa-4b68-8547-c2e18d977afd None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.473111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.475008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b447c-1d8c-43e5-ac19-e601505c6288 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.512814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.580591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad281b2-79aa-4b68-8547-c2e18d977afd None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.588617 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 310/625] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:15.590250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.590785 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.591242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.614929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:15.616388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:15.616868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.617507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.617785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.618103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.629210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.630705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:15.646420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:15.647591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 316/626] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:15.648666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.722044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 311/627] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:15.722044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.756738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.763504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:15.773377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434e65-40cf-470e-9ad1-6fd442d3a913 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.807827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.808485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.808738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.809012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.820495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.821733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:15.837379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:15.838387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:15.839400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:15.839400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:15.839400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:15.839400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:15.839400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 317/628] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:15.840116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.856046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5562d-a60f-426c-a184-150c4e6af327 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.896698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.897755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.898272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.898877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.899883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:15.900410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:15.902178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:15.915342 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 312/629] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:15.916504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:15.946385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef7f14-76c8-4c34-b764-ac7934d65f74 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:15.985997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:15.986578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:15.986856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:15.987125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:15.998568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:16.004050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:16.005144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:16.006227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:16.008204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.009934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 318/630] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:16.010676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.015114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.035112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176e99a-3dbf-4237-8ba0-75781f8e53ff None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.070749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.071497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.071866 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.072204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.073193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:16.073684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.075158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051a1e2-144c-4afb-9a83-7169df3d9ef0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.094264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 313/631] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:16.095649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dacd80-2a61-43c6-a538-a7ab303aa79a None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.096019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dacd80-2a61-43c6-a538-a7ab303aa79a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.096360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dacd80-2a61-43c6-a538-a7ab303aa79a None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.098923 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 314/632] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:16.100134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dacd80-2a61-43c6-a538-a7ab303aa79a None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.121485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608ac7c-67ba-4e1a-bf13-d5e0b994799a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.157665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.158614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.159064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.159505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.166995 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.169173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:16.183873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:16.185453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:16.186558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:16.189255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 319/633] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:16.190292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.214731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dacd80-2a61-43c6-a538-a7ab303aa79a None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.253343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.253990 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.254342 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.254684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.255464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.255591 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:16.255900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.257337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac39cd-d01f-4ee6-974f-103d52f19fdc None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.262902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.270270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 315/634] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:16.271771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.272269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.273231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.298153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:16.298579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9b33-e0df-429b-a2c1-e7e5fab814f2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.299764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:16.332746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.333387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.333668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.333966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.340442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.341913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:16.352904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:16.353997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 320/635] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:16.360321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.378273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 316/636] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:16.378273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.466244 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8416a0-3a78-4b78-af85-a5797627e637 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.466967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297f519-af6e-4a93-b31f-56795be7e776 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.488860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.495450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.500764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.501380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.501664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.501807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.501961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.502472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.502717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:16.502817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.503016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.503113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.504491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3e457-3cf2-4054-b4d8-9a688c5b84cd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.513654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.515309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.517039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 321/637] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:16.518247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4db3-a890-4f37-9fa0-c67396ddecc0 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.518533 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4db3-a890-4f37-9fa0-c67396ddecc0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.518807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4db3-a890-4f37-9fa0-c67396ddecc0 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.520187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 322/638] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:16.524640 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4db3-a890-4f37-9fa0-c67396ddecc0 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:16.531728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:16.532823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 317/639] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:16.538609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.630715 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4db3-a890-4f37-9fa0-c67396ddecc0 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.632621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6f413-d5f6-4656-aab8-ed307ca9d7e5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.665407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.666181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.666349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.666621 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.667361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:16.667668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.670155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.671997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.672628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.672906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.673176 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.681341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 323/640] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:16.682132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.687650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-2534d8c1-9687-49fd-a978-c747da7cbbf6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:16.692697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:16.693907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:16.694810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 318/641] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:16.695547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.695547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.695683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.712976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:16.714211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:16.731056 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.750463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.783816 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 319/642] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:16.784964 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.788108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a6db-1e8d-4c0a-bd70-4e8b9867d956 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.824785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.826216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.826516 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.826802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/4a1b7d1e6073474081c39f043ac1ba26/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.840975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=4a1b7d1e6073474081c39f043ac1ba26, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.842445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.854523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 324/643] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:16 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/4a1b7d1e6073474081c39f043ac1ba26/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:16.855622 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.901140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca2826-1115-459b-95ab-a9605474e3fc None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.941139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:16.941776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:16.942144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:16.942368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:16.943067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:16.943378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:16.944768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:16.958068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 320/644] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:16.962960 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:16.964720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:16.977119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d43ab3-7b22-40e3-94b4-60ffd66627d8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:16.999845 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.019239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.019870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.020152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.020753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.021283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:17.021556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.023009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.029293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 325/645] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:17.030195 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.115392 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1773-daf1-4b01-ae80-00b7cba10113 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.151553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f14bea-a4e4-4733-9360-faf98c0378ef None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.164437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.165073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.165357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.166271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.167474 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:17.167963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.171508 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.183514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 321/646] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:17.185284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.196819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.197598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.198282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.198282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.210543 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:17.216669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:17.217804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 326/647] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:17.218837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.241226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.303676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.311932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14213765-0c6e-440a-ab0d-c91fc701d7cf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.352915 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b1040-9866-4119-9800-2d43db81ca1a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.353424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.354490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.354949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.355424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.364769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.366700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:17.380533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:17.381775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 06:33:17.382958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:17.389731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 322/648] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:17.390918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.393878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.394629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.394919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.396211 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.410324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.412303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:17.432696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:17.433840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 327/649] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:17.439474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.510941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb6c1-9722-47be-83f7-c4334644f46c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.527494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.548946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da462-c948-4301-a113-308afc0633eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.550241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da462-c948-4301-a113-308afc0633eb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.550925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da462-c948-4301-a113-308afc0633eb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.551371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da462-c948-4301-a113-308afc0633eb None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.552418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da462-c948-4301-a113-308afc0633eb None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:17.553326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da462-c948-4301-a113-308afc0633eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.554892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da462-c948-4301-a113-308afc0633eb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.567175 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 323/650] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:17.568336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da462-c948-4301-a113-308afc0633eb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.574361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a765-dc3b-4fa0-bcf2-545eea4fc00e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.591676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.613149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.614092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.614527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.614961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.615937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:17.616400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.618058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.641447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 328/651] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:17.642620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.683253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da462-c948-4301-a113-308afc0633eb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.719178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.719792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.720070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.720342 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.721213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:17.721412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.723171 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.741498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 324/652] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:17.741498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.815613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3857c2-7d81-4a0c-a3ba-30030b3ec039 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.816404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.840779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fab60-c714-4321-97a5-73e6e8500602 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:17.862167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:17.876951 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.878316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.879007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.879343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:17.879909 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.880107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:17.880272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:17.880477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:17.899109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.902210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:17.930105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:17.932085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:17.934257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:17.934257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:17.934257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:17.934257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:17.937442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 329/653] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:17.939165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:17.957305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:17.958780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.039345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.066246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 325/654] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:18.067057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.123210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8ca7-f190-41a0-835e-7f30a29f7896 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.164928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.165813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.166128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.166799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.176247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9db6-dd21-4589-9135-26923dc9edf5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.177034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.179553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:18.193448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:18.194435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 06:33:18.195373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:18.199251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 330/655] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:18.199251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.220326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.220939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.221217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.221489 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.222208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:18.222501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.223901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.240289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 326/656] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:18.241136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.250454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.287852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.323198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a3a00-b070-449f-b7e0-f7bb37117369 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.359935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.361463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.362392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.362902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.378354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.379085 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319155e-602e-4946-88d9-bb2691708f5f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.380632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:18.396357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:18.397316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:18.398265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:18.398265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:18.398265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:18.398265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:18.402234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 331/657] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:18.403661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.425692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.427205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.427985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.428469 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.430163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:18.430960 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.433229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.462527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 327/658] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:18.462916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.507045 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.518714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3838d-2e3b-43e2-8e22-a0ebc2389d35 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.554588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.555410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.555696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.555971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.565448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.567100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:18.579620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:18.580616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:18.581557 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:18.585381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 332/659] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:18.586263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.597182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.607674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ebe18-3d10-4633-8dd2-7b1a520837b7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.657299 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.659100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.659554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.660006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.660875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:18.661351 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.662915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.682313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 328/660] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:18.683236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.715548 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e108ba-fd2f-45d4-b07e-378408b987b0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.755556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.756777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.757320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.758054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.783743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.800409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf4ea-7676-4f3f-afda-06334b797d05 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:18.836166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:18.836830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:18.837082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:18.837265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:18.838151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:18.838535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.839919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.845079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:18.847112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:18.863556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 329/661] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:18.864537 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.879590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:18.960921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 333/662] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:18.961567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:18.975618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00985a8a-5812-4967-83ac-71a8a9b62b1f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.011950 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.013593 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.013830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.014273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.016697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.022044 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.023527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-91926551-d5d5-4c45-b561-0f945fa12139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:19.035699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:19.036769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 330/663] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:19.038125 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f29de5-4f81-4a17-a524-e6008e20e2d7 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.038701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f29de5-4f81-4a17-a524-e6008e20e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.038701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f29de5-4f81-4a17-a524-e6008e20e2d7 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.039810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 331/664] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:19.040320 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f29de5-4f81-4a17-a524-e6008e20e2d7 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.080974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca40104-ecb2-4a1f-b9b7-261b03a12bdd None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.116756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.117018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.117379 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.117662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.117961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.152272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f29de5-4f81-4a17-a524-e6008e20e2d7 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.187981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.189440 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc50f10-40ba-45f5-9cb3-b9999f03297e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.196025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.196666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.197356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.197638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.243388 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.305629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 334/665] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:19.305629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.305629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.305629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.322419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:19.323616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:19.325820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.327347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:19.342875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:19.344002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 332/666] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:19.345107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.414222 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 335/667] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:19.415107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.447132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20529bc3-fcb8-4c1d-a8a4-d3a6be528131 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.473070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.484465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.485166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.485449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.486007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.486756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:19.487063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.488481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.495568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.500748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 333/668] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:19.501358 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.517597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606f12-bf58-4606-8e52-7b64346e6c38 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.551954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.552583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.552862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.553133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.564070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.565498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e7b49d29-3870-4756-ac1f-32fad0b7727c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:19.580889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:19.582021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 336/669] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:19.583227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb1ef8-3b67-4ed1-a708-83f3e36c7816 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.583644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb1ef8-3b67-4ed1-a708-83f3e36c7816 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.583724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb1ef8-3b67-4ed1-a708-83f3e36c7816 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.585039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 337/670] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:19.585563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb1ef8-3b67-4ed1-a708-83f3e36c7816 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.603217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44792-a434-4816-b757-7e38802cd34f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.637427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.638276 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.638558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.638833 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.639520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:19.639824 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.641186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.655128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 334/671] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:19.655970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.692199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb1ef8-3b67-4ed1-a708-83f3e36c7816 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.704092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.722363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.727944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.728586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.728872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.729158 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.739900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:19.745236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:19.746231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:19.747147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 338/672] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:19.747842 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.763210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826066-9169-4a6d-8f40-1f0d62b83d4f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.796998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.797609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.797888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.798208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.798990 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=665904332c794638bd3a890fe602cfb8) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:19.799301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.800741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503c73b-cdff-460d-8939-187a4248884f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.806346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 335/673] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/role_assignments?user.id=665904332c794638bd3a890fe602cfb8&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:19.807460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.807749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.808025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.834366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:19.835590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:19.857476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84749-ca04-4651-95c3-9235e4c89867 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:19.899334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:19.899960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:19.900239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:19.900514 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:19.901224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:19.901529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:19.903483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:19.920935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 339/674] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:19.920935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.922086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 336/675] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:19.923710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:19.981147 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:19.982996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.037933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa11cf-9f39-45af-a454-993d9d6524c8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.056597 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9b3c7-8a54-4cf4-84c2-e3158a1b40ac None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.077177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.077825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.078143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.078763 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.085114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.086575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.093680 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.094327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.094610 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.094886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.095599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.095909 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.097261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:20.098576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:20.099654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:20.100573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 340/676] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:20.101077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.111322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 337/677] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.112458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.209922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2df72b-bad8-4566-bc62-4c8dde68f4db None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.221205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.223854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.226172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accbfd6-0243-4ff5-b43f-e1eaf1f21db5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.253568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.254262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.254547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.254825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.263660 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.264541 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.265131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.265308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.265585 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.266017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.277045 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:20.279313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:20.280204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 341/678] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:20.286179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:20.286774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:20.287641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 338/679] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:20.288536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.400677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b4a2-8e00-4af8-a06e-b42bcb14281c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.412793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbc931-b50d-461c-8cb6-48d9215684dd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.437157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.437772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.438079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.438356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.439097 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.439406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.440812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.449864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.450541 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.450980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.451260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.457613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 342/680] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.457613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.468291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.469747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:20.484524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:20.485511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 339/681] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:20.486838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.493439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.505193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.565254 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d184-dfa7-4a69-924c-4908b5d2a0bc None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.602734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.603360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.603639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.603910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.604655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.604960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.606555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.609356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364fa5e-b6e2-44f2-8d07-4d6dc7d80113 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.626178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 343/682] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.626178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.650227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.651074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.651464 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.651745 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.652459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.652762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.654882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.663430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 340/683] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.668612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.729236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.739422 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaceb8-741f-4b4e-b1c1-0607be0e73fb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.745450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:20.771469 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcf73a-7400-46a4-9037-8b6f076c45a7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.776208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.776833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.777112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.777391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.778223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=cb171d91fc464ac1ae2ade235f5c7608) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.778752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.780064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.797979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 344/684] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/role_assignments?user.id=cb171d91fc464ac1ae2ade235f5c7608&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.797979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.820968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.822252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.822535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.822952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.823816 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.824128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.827034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:20.848410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 341/685] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:20.849319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:20.933437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69b5d-95c4-4351-9478-7528ac741a54 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.970441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f0329-b9c1-44b4-a8cb-fcf9f21b5bc4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:20.980895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f675-1769-4125-801e-493af1839250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:20.980895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f675-1769-4125-801e-493af1839250 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:20.981753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f675-1769-4125-801e-493af1839250 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:20.982075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121f675-1769-4125-801e-493af1839250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:20.982899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121f675-1769-4125-801e-493af1839250 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b9f7fa9ae17a4ed7a717ed044eabca83) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:20.983213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121f675-1769-4125-801e-493af1839250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:20.984655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121f675-1769-4125-801e-493af1839250 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.002272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 345/686] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/role_assignments?user.id=b9f7fa9ae17a4ed7a717ed044eabca83&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:21.003290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f675-1769-4125-801e-493af1839250 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.017395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.018112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.018395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.018670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.022270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.032207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.034049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.039724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:21.056156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:21.057149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:21.058423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:21.058423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:21.058423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:21.058423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:21.058423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 342/687] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:21.063094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.173564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121f675-1769-4125-801e-493af1839250 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.189046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822d33-755c-4305-862d-671ce7c5472b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.242664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.243262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.243975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.244722 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.245106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.245483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.245855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.246247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.247614 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:21.248602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.250286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.258385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.260016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:21.273827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:21.275049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:21.276414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:21.277101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 343/688] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:21.279550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 346/689] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:21.283047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.283886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.345380 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.376904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.398246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dde7d7-2dc5-47e3-b2fa-dde7a7df7530 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.420030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851db1a4-2782-4909-be93-fb492a3346f9 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.442387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.443558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.443845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.444123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.444924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:21.445236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.448780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.469302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 347/690] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:21.472404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.473055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.473340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.473615 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.474878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.485518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.487255 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:21.500318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:21.501519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:21.502811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 344/691] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:21.509704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.634666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f7cc8-c899-4a03-aa51-6aad055f5cc7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.700507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.701066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.701351 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.701626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.702236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1335d7b-71ea-4a08-9961-e1b584bb25aa None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:21.710292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.716181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:21.752475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:21.753158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:21.753446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:21.753723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:21.755216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:21.755531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.757168 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.787805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 345/692] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:21.795748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:21.829901 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:21.832421 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4eab92-e4da-43ff-9d11-fa301dce4a02 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:21.946337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77af1e-b5e8-4b04-b200-38f17e79ea1e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.001503 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.002317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 348/693] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 06:33:22.003665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a57e4-6d78-4556-9e9b-0e741f36c636 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.003960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a57e4-6d78-4556-9e9b-0e741f36c636 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.004238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a57e4-6d78-4556-9e9b-0e741f36c636 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.005755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 349/694] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:22.006129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.006594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.006974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.011325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.011666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.011949 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.034855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:22.036069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:22.048305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.078507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.101422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.104203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.133980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 350/695] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:22.133980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.235818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 346/696] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:22.235818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.256427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c8f2b-61c7-4ea5-8869-806fa3029f29 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.294218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.295215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.295848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.296325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.296794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.310419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.312219 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:22.328252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:22.329166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:22.330106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:22.330106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:22.330106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.330106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:22.336331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b0da2-e4eb-44ee-a058-8e2a215d2e75 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.336920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 351/697] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:22.338062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.366381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.372599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.373039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.373401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.373693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.384276 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.385738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:22.403759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:22.404677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:22.405573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:22.405573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:22.405573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.405573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:22.409896 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 347/698] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:22.410735 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.454419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b1a4-8080-4001-9926-c806d0ba7670 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.492704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.493049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.493323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.493600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.504488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.505976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:22.526168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:22.527143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:22.528178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:22.528178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:22.528178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:22.528178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:22.531659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 352/699] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:22.532609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.538013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df6b5-cdfd-4d20-ba78-3b27eb2badad None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.560822 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.586848 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.588250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.588741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.589222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.590912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:22.591436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.593529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.608830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.611157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 348/700] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:22.614232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.657990 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d66fa3-fba1-449b-8256-04fbac2b9b6d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.701393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.702428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.702913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.703343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.704185 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:22.704732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.706305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.719577 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 353/701] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:22.720766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.750967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ce908-241a-4f9a-bbd8-b03d9dee9be7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.789307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.789852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.790161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.790438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.791334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:22.791558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.792936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.805991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 349/702] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:22.806512 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.831241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c1e88-b086-4531-b157-caa3fa2c9550 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.844863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.862887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:22.868688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.869394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.869832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.870149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.877427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.878994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:22.890604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:22.891541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:22.892454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 354/703] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:22.892947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:22.926119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774792ed-ed40-4685-9cc3-4ce3e34b1ed6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:22.962796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:22.963523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:22.963809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:22.964088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:22.964766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:22.966837 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:22.994830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 350/704] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:33:22 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:22.995665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.022364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bfe4f-a842-4f22-906e-951ef543f525 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.091768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.092484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.092767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.093044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.107759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.110800 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:23.127967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:23.130157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:23.132294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 355/705] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:23.137981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.179986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.191498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0f88-2feb-4216-9539-7749b952908f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.203846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.236245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.236872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.237153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.237435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.238234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:23.238551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.240056 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0108f-fe3e-47a5-9421-1115036e5fc8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.254055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 351/706] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:23.255307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90a662-a7fa-4c50-8fb3-c82559d3ca34 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.255598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90a662-a7fa-4c50-8fb3-c82559d3ca34 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.255874 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90a662-a7fa-4c50-8fb3-c82559d3ca34 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.257210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 352/707] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:23.261788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90a662-a7fa-4c50-8fb3-c82559d3ca34 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.272714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16864-4fa2-45f3-8150-ffd24f81f2d4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.319164 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.319792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.320077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.320355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.321112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:23.321423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.323023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53421f3-affa-47ba-b4cc-5cbe3d48bdc1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.338300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 356/708] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:23.339877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb41-5093-4bb0-9ed8-e3717f9c0ba4 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.345208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb41-5093-4bb0-9ed8-e3717f9c0ba4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.345847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40eb41-5093-4bb0-9ed8-e3717f9c0ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.364860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-dd40eb41-5093-4bb0-9ed8-e3717f9c0ba4 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:23.366624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-dd40eb41-5093-4bb0-9ed8-e3717f9c0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:23.386153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90a662-a7fa-4c50-8fb3-c82559d3ca34 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.425376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.426024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.426321 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.426596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.427377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=4a1b7d1e6073474081c39f043ac1ba26) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:23.427749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.429104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.436131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 353/709] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/role_assignments?user.id=4a1b7d1e6073474081c39f043ac1ba26&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:23.437135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.442659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 357/710] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:23.467093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.474108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304438-a31c-4d32-b9d4-9210d882bba6 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.474711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304438-a31c-4d32-b9d4-9210d882bba6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.475192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304438-a31c-4d32-b9d4-9210d882bba6 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.477102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 358/711] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:23.478333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304438-a31c-4d32-b9d4-9210d882bba6 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.528747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.566047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e976-bf5d-44e7-81b0-7b81ec048463 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.594189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304438-a31c-4d32-b9d4-9210d882bba6 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.605753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.606788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.606934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.607209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.608013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:23.608328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.611376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e04bf-a1d1-4d4d-b8b9-76f626c8b7f1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.618163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 354/712] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:23.619138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.619649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.619930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.646770 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:23.648010 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:23.669224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.671015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.671672 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.672172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] PATH_INFO: `/v3/services/network` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.673201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.675670 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:23.694530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:23.695705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:23.696714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:23.702961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 359/713] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:23.707946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.750332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 355/714] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:23.772572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.786836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.835726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773faf-f163-4d5d-8cf3-6fd5299a2f46 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.845114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:23.883933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.884483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.884768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.885043 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.885728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] RBAC: Adding query filter params (name=network) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:23.886274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.887944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.898101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 360/715] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:23.905309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:23.906374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcafc6-6ea2-40d5-8a48-6c6fd53948a2 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:23.947563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:23.948217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:23.948501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:23.948778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:23.949771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:23.951446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:23.964348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:23.965416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:23.966754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 356/716] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:23.979171 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:24.023685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07278-243d-433b-a4a5-f027374a9f5e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:24.061426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:24.062087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:24.062374 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:24.062652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:24.063333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] RBAC: Adding query filter params (type=network) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:24.063659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:24.065396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:24.072222 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 361/717] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:24.162314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:24.198157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862300e2-5765-4ec9-aad9-1eca3df116a8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:24.250758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:24.251407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:24.251685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:24.251955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:24.252654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:24.252953 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:24.254358 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4279-1aca-4474-b4b6-e9c9eaa87681 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:24.260515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 357/718] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:24.267230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:24.301733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:24.370815 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3c7b-05c1-4c81-b4e6-5e6af26200ab None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:24.417538 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:24.418787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:24.419324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:24.419802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:24.420685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:24.421215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:24.422820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08f2b-7147-4e20-9b29-67446800093a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:24.428000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 362/719] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:25.612997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b1933-ea18-45c3-ae9c-2c00f79dc3ee None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:25.613525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b1933-ea18-45c3-ae9c-2c00f79dc3ee None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:25.613525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b1933-ea18-45c3-ae9c-2c00f79dc3ee None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:25.615156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 358/720] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:25.620627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:25.621042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:25.621235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:25.640980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:25.642199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:25.730058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 363/721] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:25.750713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b1933-ea18-45c3-ae9c-2c00f79dc3ee None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:25.894717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b1933-ea18-45c3-ae9c-2c00f79dc3ee None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:25.935221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:25.935921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:25.936207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:25.936485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:25.946547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:25.951032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-058ef774-142b-41fb-995d-82e7ce4fb8bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:25.955979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:25.956906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:25.957763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:25.961765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 359/722] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:25.963405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.001814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.093764 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebc6e4-5aa8-4aa2-aa17-66cd9ba9df6f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.095255 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ef4e-dc15-477b-9058-d285b6ef5954 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.095545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ef4e-dc15-477b-9058-d285b6ef5954 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.095820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ef4e-dc15-477b-9058-d285b6ef5954 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.098228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 360/723] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:26.102218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a44d-8f64-4920-923a-35a1983db9cb None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.102504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a44d-8f64-4920-923a-35a1983db9cb None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.102780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a44d-8f64-4920-923a-35a1983db9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.138174 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-da48a44d-8f64-4920-923a-35a1983db9cb None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:26.139165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.139417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-da48a44d-8f64-4920-923a-35a1983db9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:26.140246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.140729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.141221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.142181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:26.142692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.144275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.153755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 364/724] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:26.159512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.240775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 361/725] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:26.242363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c2f4-921c-4b55-b367-ac0f56d02864 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.242647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c2f4-921c-4b55-b367-ac0f56d02864 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.242922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c2f4-921c-4b55-b367-ac0f56d02864 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.248441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 362/726] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:26.250211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.250489 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.250774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.270598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:26.271772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:26.283962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.294503 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c073bb-0bc1-4589-a7e1-e9aa12a2bbcf None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.329250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.330292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.330782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.331265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.342785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.344502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.354026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 363/727] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 06:33:26.355023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:26.360279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:26.361441 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:26.362406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:26.362406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:26.362406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:26.362406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:26.364553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 365/728] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:26.370414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.435124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.474856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b42c-e894-404a-a4a7-84577c8609c0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.480941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bc501-7f20-4d77-9d20-ae9f304c3eae None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.510786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.511572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.512026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.513071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.513870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:26.514298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.515936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.520243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.521057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.521501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.521914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.533475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:26.533652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 366/729] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:26.534617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.537430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:26.540398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:26.541346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:26.542249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:26.545359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 364/730] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:26.546448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.639492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98dc0-c88b-4355-89ce-f484fa69b907 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.657482 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bca89b-ffa0-4567-a7e6-b76288273cd1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.678389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.686260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.686839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.687222 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.687473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.697538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.697962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:26.698203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.698481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.698753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:26.702933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:26.703873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:26.704729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 367/731] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:26.705438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.711267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.712802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a6909fc7-c395-454c-b27e-5216db500b47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:26.726336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:26.727348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 365/732] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711036-a669-4a00-bd08-f664993f3ef3 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.734333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711036-a669-4a00-bd08-f664993f3ef3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.735041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711036-a669-4a00-bd08-f664993f3ef3 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.735041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 366/733] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:26.736653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54711036-a669-4a00-bd08-f664993f3ef3 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.778556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.823313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb74fb-1268-4ae1-9e3a-9383e4b4f225 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.844702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54711036-a669-4a00-bd08-f664993f3ef3 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.861896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.862543 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.862829 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.863109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.863820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:26.864131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.865485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.879476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 368/734] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:26.879476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:26.885790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:26.886442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.886725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.887032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.887750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:26.888058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:26.889416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657124a-1865-43ff-b4cb-fbf3b9f8f5f2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:26.897727 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 367/735] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:26.902372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:26.902372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:26.902372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:26.917622 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:26.919533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:26.958327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:26.991041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628700-51a8-4501-a250-54e7873a4e63 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:26.994163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 368/736] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:26.994914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.028556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.040366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.041139 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.041973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.041973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.042613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:27.042919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.044941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.064369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 369/737] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:27.065206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.114587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab71c77-909c-40a3-8765-998d12fbf7a7 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.148684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.149319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.149601 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.149875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.159684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.161116 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.162037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c522d4d-c29d-46a6-9e20-90668bef260f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:27.176095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:27.177178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 369/738] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:27.178325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.200601 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.201233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.201517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.201793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.202645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.211902 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.213278 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-9a0e97a0-a40b-4e0a-ad24-4046e4974d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:27.229033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:27.230160 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:27.231165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:27.231165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:27.231165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.231165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:27.231165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 370/739] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:27.232296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53dcd5-b1b2-4ab7-b2cb-4b22108dae61 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.232579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53dcd5-b1b2-4ab7-b2cb-4b22108dae61 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.232862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53dcd5-b1b2-4ab7-b2cb-4b22108dae61 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.238233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 371/740] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:27.238233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53dcd5-b1b2-4ab7-b2cb-4b22108dae61 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.270271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.284112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f2785-155d-413a-a95c-898fd026b8f5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.317815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.318458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.318745 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.319026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.319717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.321084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.336870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 370/741] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:27 2022] POST /identity/v3/services => generated 240 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:27.337693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.357169 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53dcd5-b1b2-4ab7-b2cb-4b22108dae61 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.394051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.394694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.394973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.395255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.448624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.471508 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.472502 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531de2c-60bf-4222-8ad0-6123dec3d404 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.472878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0ea57-8cad-40f2-aa9a-3d72463ef3c2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.507725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.516313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.516943 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.517232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.517605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.518349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:27.518851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.520033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.533102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 371/742] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:27.533858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.588942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 372/743] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 06:33:27.590521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70933-3753-43bb-8848-09338242f058 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.590961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70933-3753-43bb-8848-09338242f058 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.591386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70933-3753-43bb-8848-09338242f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.610158 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-cdf70933-3753-43bb-8848-09338242f058 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:27.611617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-cdf70933-3753-43bb-8848-09338242f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:27.648213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee97bf0-3b69-4862-8835-f64bec7e5fed None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.689648 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.690318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.690669 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.702282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.702443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 373/744] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:27 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 06:33:27.702839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:27.702839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.703007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf70933-3753-43bb-8848-09338242f058 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.703082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.713898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 372/745] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:27.715062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.717973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.766561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.813979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf70933-3753-43bb-8848-09338242f058 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.824030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba1c9-9199-42fb-a498-84d5908daeb1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:27.847631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.848272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.848555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.849011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.856917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.858390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.860404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:27.861766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:27.862080 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:27.862359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:27.870519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:27.872041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 374/746] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:27.873009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.878692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:27.880139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:27.895165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:27.896124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:27.897024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:27.897024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:27.897024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:27.897024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:27.897024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 373/747] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:27.897842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:27.957660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:27.986826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1c4a-e4bb-4bae-a69a-b443c15a5a5d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.007946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedb3da-5d61-4e17-9b86-f79562844ee0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.008158 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.025069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.025894 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.026299 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.026844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.037007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.039238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.042991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.043704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.044051 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.044388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.045137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.046697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:28.052224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:28.053437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:28.054893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:28.058594 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 375/748] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:28.059939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.062719 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 374/749] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:27 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:28.063596 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.193284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe595-809b-4fbf-a14d-f7b400311204 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.208501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530091-d253-485b-be7f-b017fd1e9861 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.222163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.237874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.238560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.239571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.239571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.239886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:28.240188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.241725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.257211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.257841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.258150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.258429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.259128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:28.259434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.260795 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.266134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 376/750] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:28.266134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.277465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 375/751] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:28.278925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.315473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.392122 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972a8c-2e63-4274-a83a-bd9b4b2855e4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.392324 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734394ca-9b21-45c8-ac64-ea9f93d0d065 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.431404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.432451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.432938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.433439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.484331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.518106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.519926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.556836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.557049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.557274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.557457 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.558291 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:28.558651 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.560177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.579279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 377/752] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:28.580332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.628985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 376/753] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:33:28.630254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.705822 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.713718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da501a2-4e19-4ccc-9b47-8f8398b8253c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.726101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:28.769214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f25002-d97b-462a-9bfc-7b91bff3b825 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.769741 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.770684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.771190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.771670 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.779708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.781458 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:28.793546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:28.794500 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:28.795412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:28.797827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 378/754] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:28.798931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.814685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.815259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.815540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.815819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.830220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.831962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:28.851228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:28.852212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:28.853113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:28.853113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:28.853113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:28.853113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:28.853113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 377/755] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:28.858493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:28.924830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf7041-3294-4690-883b-0310dad0aa7b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.966749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:28.967678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:28.967963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:28.968242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:28.979546 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8936-559e-4208-80e6-d55cbb357d73 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:28.983495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:28.984952 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:29.004600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:29.005560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:29.006507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:29.006507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:29.006507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:29.006507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:29.006507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 379/756] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:29.007229 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.010529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.023369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.030786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.031839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.032279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.032978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.034294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.034764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.036875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.059951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 378/757] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:29.059951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.126402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe7b26-9ae2-4231-8c65-638ba97304f6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.161623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.162621 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.163102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.163605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.164484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.164993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.166631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.170792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656cec3-1fba-46d5-b6ac-353b630c96c5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.179904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 380/758] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:29.181113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.207427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.207894 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.208180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.208457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.209146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.209506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.210961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.227685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 379/759] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:29.227685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.270334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.285091 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.307420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b329dd8-c4eb-4318-ab23-b87b05d08387 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.342586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.343590 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.344085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.344586 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.345566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=793cd10dcc4344f2be80fd34a82c0d6e) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.346140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.347750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.362090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 381/760] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/role_assignments?user.id=793cd10dcc4344f2be80fd34a82c0d6e&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:29.363225 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b431df-0374-4eea-ba8d-f45af962fbf5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.363633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.411284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.412419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.412703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.412987 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.423674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.425163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-229dd150-84f9-4596-ae62-38bac65a3396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:29.442767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:29.443754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:29.445118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 380/761] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:29.448627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02db8a-1dab-45b6-9724-0d9216ef29d5 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.448627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02db8a-1dab-45b6-9724-0d9216ef29d5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.448627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02db8a-1dab-45b6-9724-0d9216ef29d5 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.449225 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 381/762] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:29.450319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02db8a-1dab-45b6-9724-0d9216ef29d5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.483419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a60ab8-5eab-414e-bbe0-604139e422d4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.521922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.522845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.523129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.523705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.531139 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.532840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-37b15eaf-1701-429c-9e04-7415c859d127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:29.546080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:29.547026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:29.547932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:29.554029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.554573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 382/763] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:29.556059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.556564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.557037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.576848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:29.578501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:29.580109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02db8a-1dab-45b6-9724-0d9216ef29d5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.582627 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.616457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.617073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.617366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.617621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.618393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.618698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.620114 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.639748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 382/764] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:29.640841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.677966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 383/765] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:29.677966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.787286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b021-c12f-46a1-baf9-57bd0607af0f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.794237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27e34-1c87-4636-84f7-92e05d3adc61 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:29.825321 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.825934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.826076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.826256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.826533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.827828 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:29.828072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.829536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.841688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:29.842904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:29.843333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:29.843783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:29.844628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 383/766] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:29.845704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.846322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=b426ebd709e942819f19926f2e3668d2, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:29.847920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:29.870711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 384/767] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:29 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:29.872117 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:29.882698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:29.994884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6ba84-6af6-4502-bd07-5032637f0b93 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.035301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.036216 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.036629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.037034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.038010 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=4a1b7d1e6073474081c39f043ac1ba26) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:30.038474 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.040085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.040375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8fde-d0c5-483f-b746-3f0926532110 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.047595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 385/768] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/role_assignments?user.id=4a1b7d1e6073474081c39f043ac1ba26&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:30.118221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.118880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.119158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.119433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] PATH_INFO: `/v3/services/network` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.120067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.121481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-05e58aa0-3313-49a8-8082-ad991af3d87f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:30.135190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:30.136442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.138038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 384/769] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:29 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:30.145083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.204786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:30.265840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb327b-4c8f-462e-b8e4-ed184caa21a8 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.266165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb327b-4c8f-462e-b8e4-ed184caa21a8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.266347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb327b-4c8f-462e-b8e4-ed184caa21a8 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.266670 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9021d8-d331-47de-b641-69de63dc3cbc None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.267808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 385/770] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:30.276511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.276944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.277318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.307606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:30.309572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:30.313977 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.314509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.314844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.315123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.315807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] RBAC: Adding query filter params (name=network) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:30.316119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.317480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.330054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 386/771] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:30.335264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.395461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 386/772] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:30.422448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.444214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:30.540755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d270b-d65a-478b-8334-8ae16f9de056 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.567458 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9caf-2e1a-431d-b09f-efbe57d2528d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.580638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.582060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.582988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:30.583637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.584152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.585036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.586698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:30.599260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:30.600456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:30.601524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:30.604322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 387/773] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:30.611205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.644159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.644856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.645115 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.645435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.646402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] RBAC: Adding query filter params (type=network) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:30.646523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.647889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.659314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 387/774] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:30.664585 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.738229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e821-82a2-4de8-a9d7-03b88efefd52 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.741397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:30.775462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.776349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.776838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.777301 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.778254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:30.778711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.780238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.785598 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 388/775] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:30.791613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:30.868125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71efb-f708-4526-9dd8-046a75e86f13 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:30.937220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:30.937893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:30.938201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:30.938475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:30.939230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] RBAC: Adding query filter params (interface=public, service_id=6a8c228d2bc240938cb7ef700b31202a, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:30.939539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:30.940889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:30.949282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 388/776] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/endpoints?service_id=6a8c228d2bc240938cb7ef700b31202a&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:30.955750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:31.020531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d89d7-c424-4b54-95c5-7c4d861ca07c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:31.029708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:31.097136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:31.097797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.098100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.098374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.099049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:31.099353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:31.100713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:31.115356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 389/777] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:31.123046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:31.137536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:31.151054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544066b-aa44-4a01-a771-fbd6de22492d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:31.222655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:31.223281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.223553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.223824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.224478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:31.225830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:31.232201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 389/778] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:30 2022] GET /identity/v3/services => generated 772 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:31.282338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3a96f-5fd9-4205-aa92-a373646438de None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:31.333584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:31.334266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.334550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.334827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.335606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] RBAC: Adding query filter params (interface=public, service_id=d64520cdcdc24aac8428f88a9e73044b, region_id=RegionOne) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:31.335920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:31.337329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54800d61-b2b3-4d1d-900c-9e9c614bcb74 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:31.346835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 390/779] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:31 2022] GET /identity/v3/endpoints?service_id=d64520cdcdc24aac8428f88a9e73044b&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:31.352461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:31.507487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416b55-2138-4fe1-92b6-a1e1170322ed None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:31.509040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:31.542560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:31.543800 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.544293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.544568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.545217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:31.546647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecc5fb-9bc9-4d2b-9f40-5c3d8f46bd34 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:31.552240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 390/780] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:31 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:31.644892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cbb46-4a61-4b2c-b1aa-f172ba7d727e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.645087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cbb46-4a61-4b2c-b1aa-f172ba7d727e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.645374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cbb46-4a61-4b2c-b1aa-f172ba7d727e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.646770 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 391/781] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:31.653696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.654006 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.654286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.678019 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:31.679194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:31.763486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 391/782] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:31.785406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cbb46-4a61-4b2c-b1aa-f172ba7d727e None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:31.880619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:31.924077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cbb46-4a61-4b2c-b1aa-f172ba7d727e None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:31.972102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:31.973151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:31.973674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:31.974218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:31.985643 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:31.990941 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:31.991877 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:31.992737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:31.995424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 392/783] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:32.007133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.109981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.119694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ecfcc-8997-4c47-b0e9-8ad762c034be None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.161433 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:32.162108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.162397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.162681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.163411 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:32.163726 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:32.165093 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f79c4e-0b4f-4c6c-8eed-7d3bf737ec49 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:32.178833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 392/784] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:32.187022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.336101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e4ad-0330-471b-aad4-281dfbea96d5 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.377175 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.382509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:32.383352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.383700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.386284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.391199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bca8-dd4c-4b7f-8342-e6807041655a None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.391491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bca8-dd4c-4b7f-8342-e6807041655a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.391770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bca8-dd4c-4b7f-8342-e6807041655a None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.395443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 393/785] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:32.399286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:32.400747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:32.404985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230aa1b-66ca-41d2-bb79-f782cf9dba25 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.405284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230aa1b-66ca-41d2-bb79-f782cf9dba25 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.406060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230aa1b-66ca-41d2-bb79-f782cf9dba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:32.432517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:32.433446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:32.434872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:32.434872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:32.434872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:32.434872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:32.434872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 393/786] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:32.440295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-4230aa1b-66ca-41d2-bb79-f782cf9dba25 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:32.441504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4230aa1b-66ca-41d2-bb79-f782cf9dba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:32.445262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.532513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 394/787] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:32.533870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae4b5e-cdec-48b6-a4f5-d145b15d2d2c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.534207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae4b5e-cdec-48b6-a4f5-d145b15d2d2c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.534486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae4b5e-cdec-48b6-a4f5-d145b15d2d2c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.535996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 395/788] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:32.540946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.541235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.541512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.571306 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:32.573192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:32.578854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdd5f-b8c5-4120-b438-6411a8bc4db6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.614998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:32.615666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.615954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.616237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.617030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:32.617350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:32.618743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:32.631963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 394/789] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:32.632922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.658798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 396/790] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:32.659806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.682196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.747096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.779544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5a31c-9f51-4028-9e0d-efc6087d6a41 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.782288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486cdd-cc85-43a6-93a0-7899a0ae5766 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.817516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:32.818291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.818574 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.818852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.830390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:32.831004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:32.831650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:32.831929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:32.832249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:32.835977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:32.836913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 395/791] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:32.838452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.841825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:32.841825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:32.859568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:32.861266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:32.862578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 397/792] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:32.863332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:32.954167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cc07-792e-43d9-92a8-d7dcc92ea844 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:32.957429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.971591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:32.998429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1079c0-eb6f-4002-a62c-4dd2fca8da03 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.008456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.010063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.010552 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.011042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.026860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:33.034012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.035002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:33.035839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:33.036475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.036475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.036475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.036475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.037074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.037074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.038090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.045136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 396/793] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.045136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.055123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 398/794] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.056512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.162182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3143-ce58-45bc-a066-35a8367eee99 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.184890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b71a48-744c-4fd0-be8d-893f482b0f2d None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.209595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.210750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.211203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.211644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.213243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.213830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.216727 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.222561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.223304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.223689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.224067 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.224879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.225287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.226747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.241280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 399/795] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.241280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.244066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.246749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 397/796] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.249061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.281543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.362744 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c9eb2-c5cc-4812-9aba-f6fa71411290 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.377810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9ac-b0c2-4f41-8fa3-951e02f3f631 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.403016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.404226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.404736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.405193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.415173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.415914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.416349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.416541 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.417852 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.419642 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.428354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.429880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:33.436956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.437845 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.438793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:33.438793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:33.438793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.438793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:33.441126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 400/797] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.442159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:33.447084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.447984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.448904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:33.448904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:33.448904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.448904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:33.453336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 398/798] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.454695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.506289 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.537873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.559614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a3e09-7652-4f65-9c7b-61cabe98068a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.566708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d298018-cbea-4f1d-ba21-bd20c147ea88 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.596927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.597874 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.598448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.598933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.604546 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.605167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.606439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.606859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.607563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.607872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.609344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.613217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.614964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.623202 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 399/799] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.624268 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:33.630973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.631930 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.632838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:33.632838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:33.632838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:33.632838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:33.640118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 401/800] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.641426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.737013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6673af7-757c-4494-a4f0-e14890d92d91 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.760820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05091ca7-a9e4-413b-b10b-dcfcfdb3e2b0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.768655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.778327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.779266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.779697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.780201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.781140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.781650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.783778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.785750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:33.797880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.798553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.798833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.799097 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.799419 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 400/801] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.800542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.811947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.813610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6253d167-744b-4067-a347-93963a0b4616 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:33.826946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.828089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:33.829031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 402/802] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.829729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380de39e-9215-4deb-bcf0-6edef9d71544 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.830041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380de39e-9215-4deb-bcf0-6edef9d71544 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.830323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380de39e-9215-4deb-bcf0-6edef9d71544 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.831676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 403/803] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:33.832610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380de39e-9215-4deb-bcf0-6edef9d71544 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.919976 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d58af6-57b5-4e5e-92fa-8844ae988bbf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.940280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380de39e-9215-4deb-bcf0-6edef9d71544 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:33.955235 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.955852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.956130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.956404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.957096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:33.957388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.958786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.971132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 401/804] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:33.971776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:33.974569 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:33.975438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.975553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:33.975739 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:33.983251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:33.984672 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c23447fb-a2ad-4bea-a029-0fef035f6e21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:33.997037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:33.998286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 404/805] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859990-c3c5-4fb0-bcb5-5020a6d040b6 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:33.999449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859990-c3c5-4fb0-bcb5-5020a6d040b6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.000322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859990-c3c5-4fb0-bcb5-5020a6d040b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.017833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-4f859990-c3c5-4fb0-bcb5-5020a6d040b6 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:34.018967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4f859990-c3c5-4fb0-bcb5-5020a6d040b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:34.030678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.039477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.077445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf5cf1-425f-4315-8254-346d225a85ac None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.084889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 405/806] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:34.086163 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d908fe-2464-4f83-9d11-2f28b4efb56b None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.086450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d908fe-2464-4f83-9d11-2f28b4efb56b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.086724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d908fe-2464-4f83-9d11-2f28b4efb56b None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.087939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 406/807] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:34.088620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d908fe-2464-4f83-9d11-2f28b4efb56b None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.118114 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.118730 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.119008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.119278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.120059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:34.120366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.122774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.140143 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 402/808] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:33:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:34.141018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.191103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d908fe-2464-4f83-9d11-2f28b4efb56b None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.223813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.224444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.224721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.224994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.231561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.232975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.237691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f74f5-9bb0-4544-9e04-0c9619ea5295 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2c85e08d-188a-40ca-b609-fc492dc77acc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:34.244005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:34.245000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 407/809] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.247322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.247961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.254647 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.265402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:34.266645 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:34.276762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.277422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.277607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.277705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.278102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.278902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:34.279217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.280796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.297747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 403/810] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:34.298663 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.336232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 408/811] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:34.337079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.412479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e471b-4088-41f8-a705-c69fb4b53463 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.435218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb006f-1d12-4ec7-b032-9c39506e07ae None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.446194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.446831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.447111 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.447387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] PATH_INFO: `/v3/services/network` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.448012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.449891 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:34.458728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:34.459702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:34.460587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 404/812] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:34.461389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.472562 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.473187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.473476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.473760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.489962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.497106 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.542309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.544558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.562280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52a80a-c6d3-4098-9e10-09427e6f5b97 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.595248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.595852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.596132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.596406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.597142 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:34.597448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.598886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.610986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 405/813] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:34.611755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.647429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 409/814] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:33:34.648271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.699834 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.710394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.717806 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0795b-3bf9-4e89-b20b-0375f3bcae67 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.751705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.752340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.752618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.752892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.761337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6621-6b2e-46a7-9dfd-d94dee0a28af None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.763205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.764634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:34.779006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:34.780075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:34.780974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:34.780974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:34.780974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:34.780974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:34.780974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 406/815] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:34.781247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.799204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.799790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.800069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.800343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.800968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.802348 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:34.811546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:34.812525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:34.813399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 410/816] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:34.814301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.884826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c64c2-a267-45c4-8333-b0838e72056e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.911948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6be6-178d-4ee9-88a3-554e9caf27d2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:34.917214 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.918379 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.918690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.919081 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.919753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] RBAC: Adding query filter params (name=network) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:34.920060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.921424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.922302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.926193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 407/817] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:34.926973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:34.942067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:34.946646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:34.947312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:34.947591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:34.947869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/793cd10dcc4344f2be80fd34a82c0d6e/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:34.962567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=793cd10dcc4344f2be80fd34a82c0d6e, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:34.963984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:34.977011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 411/818] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:34 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/793cd10dcc4344f2be80fd34a82c0d6e/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:34.977817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.041123 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45248-8658-4cd1-a987-31842b204642 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.083060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.084143 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.084623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.085054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.097275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.098776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.099658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a7be-7f06-4f47-a88a-cf4b9580fed6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:35.115414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:35.116398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:35.117306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:35.117306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:35.117306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:35.117306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:35.117306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 408/819] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:35.117594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.147751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.148380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.148664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.148948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.149645 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.149982 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.151349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.166554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 412/820] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.167405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.191856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.194256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.238963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26bc9-c85e-46dd-b233-f59c6c849955 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.275160 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.275782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.276068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.276347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.277028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.277440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.278074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b607ea-f1cf-4287-afd3-323409f2ca6b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.278836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.283583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 409/821] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.284375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.319488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.320106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.320394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.320676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.321353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] RBAC: Adding query filter params (type=network) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.321668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.323073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.333004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 413/822] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.333004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.401814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad446d-e73e-471e-bd8c-6b0df108810d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.419783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.429332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.441149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843a39-0634-478e-8b96-1965c9621b49 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.441586 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.442311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.442600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.442878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.443577 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.443884 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.445246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.459227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 410/823] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.460326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.482607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.483402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.483683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.483950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.491880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.493353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:35.506495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:35.507439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:35.508341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 414/824] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:35.508826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.567862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93f27c-e746-4b33-9914-b087cf86c8ff None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.603118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.603738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.604023 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.604300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.604981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.605503 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.606758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.611855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 411/825] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.612635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.620472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167edcc-78d3-4e42-a031-6b2176215199 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.662945 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.664420 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.664705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.664980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.665690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.667086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.671875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.686145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.693335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 415/826] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:35.694400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.751739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b673b56-5c0a-41b6-8969-3afe0450d4a2 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.815310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.816413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.816829 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.817203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.828679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.832339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.840439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4bcec-a20b-465f-ba4e-efbd7af9d1d9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:35.849458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:35.850412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:35.851318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:35.854028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 412/827] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:35.854857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.894189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:35.894823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:35.895108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:35.895386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:35.896141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:35.896791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:35.897875 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:35.913233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 416/828] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:35.914495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:35.979876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:35.988349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc126e4-69d8-4905-bc2e-e00b26c9fd40 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:35.996553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:36.024147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1230846-7aaa-4dba-881e-82c5bebd374a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:36.028781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:36.029401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:36.029683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:36.029981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:36.030690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:36.032512 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:36.059610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:36.060293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:36.060605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:36.060881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:36.061614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:36.061923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:36.063372 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:36.064412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 413/829] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:36.074170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:36.086666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 417/830] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:36.093036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:36.204418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf7119-e8d5-4c68-8ca2-1b9012c6dc60 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:36.230296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3ce65-d31c-487c-b3a9-92015df51cd8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:36.250553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:36.251578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:36.252075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:36.252549 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/4a1b7d1e6073474081c39f043ac1ba26/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:36.253737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:36.255207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=4a1b7d1e6073474081c39f043ac1ba26, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:36.257180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900ed8c-ebee-45d6-9f2b-e1a0bae2ad92 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:36.270285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:36.271364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:36.271364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:36.271536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:36.272794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:36.272794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:36.273925 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95307da-ecf4-4192-9e8e-9bced09655b2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:36.277029 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 414/831] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:36 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/4a1b7d1e6073474081c39f043ac1ba26/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:36.281989 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 418/832] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:36 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:37.005920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a0aa-098b-4924-979f-bb0bd0aff10d None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.006612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a0aa-098b-4924-979f-bb0bd0aff10d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.007098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a0aa-098b-4924-979f-bb0bd0aff10d None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.008871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 415/833] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:37.014357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.014885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.015362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.034730 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:37.036235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:37.105290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 419/834] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:37.129082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a0aa-098b-4924-979f-bb0bd0aff10d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:37.148772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:37.233016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a0aa-098b-4924-979f-bb0bd0aff10d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:37.267970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:37.268540 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.268819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.269093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.280613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:37.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:37.286634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:37.287924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 416/835] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:37.292663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:37.397427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670e2a1-c943-4866-a3cb-3b45a1f48e24 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:37.412969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:37.439262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:37.440238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.440718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.441189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.442143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:37.442663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:37.444208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:37.454377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 420/836] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:37.460368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:37.550607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:37.570494 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd280-7a20-43a4-8d9b-5815eb097fd1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:37.607076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:37.607723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.608004 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.608283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.618681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:37.620119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:37.635049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:37.635924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:37.637492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:37.637492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:37.637492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:37.637492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:37.637492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 417/837] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:37.643807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:37.766302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485276d-6681-438e-bd25-a2d77316b5c8 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:37.801275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:37.801817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.802128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.802405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.803092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:37.803397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:37.804743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:37.817981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 421/838] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:37.823358 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:37.927625 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096fedf-1c7b-4c52-be14-3ff60ebc53bb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:37.962345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:37.964090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:37.966036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:37.966564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:37.967210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:37.975587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:37.977307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:37.989367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:37.990346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:37.991284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:37.993765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 418/839] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:38.000221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.036652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.108378 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3323f-9896-4924-8d83-6fca91a0909b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.144783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.145770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.146310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.146783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.147779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:38.148368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.150069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14b498-f60a-4fbd-a819-40726dacd5ef None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.169018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 422/840] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:38.174817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.178021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a18258-1982-4a27-9b86-0ffb21031b75 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.178320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a18258-1982-4a27-9b86-0ffb21031b75 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.178598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a18258-1982-4a27-9b86-0ffb21031b75 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.180725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 423/841] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:38.185541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.185824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.186132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.204550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:38.205839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:38.281885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 424/842] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:38.294845 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7e551-b2ba-4a89-b993-e5bd0297bc86 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.304382 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.329889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.336504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.337370 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.337847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.338365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.399833 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.410499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538223e-a6e7-4fb7-9153-311a3d7756b5 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.411730 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.413189 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.451157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.452173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.452665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.453616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.454501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.456893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e58dfae8-690a-4673-855d-9b28f4f2f96b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:38.469472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:38.470675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:38.471712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:38.472960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 425/843] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:38.473788 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b85b86-1f2d-4400-b920-bc2951ff45a1 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.474122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b85b86-1f2d-4400-b920-bc2951ff45a1 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.474406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b85b86-1f2d-4400-b920-bc2951ff45a1 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.475808 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 426/844] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:38.481385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.481689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.481986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.500258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:38.501457 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:38.524970 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 419/845] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:33:38.526087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.569818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 427/846] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:38.570628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.588986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.630513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.640573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda7080-53e4-412b-862b-8e15c737a182 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.673103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757207c-4ce6-41b9-a46e-119a1157e561 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.675052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.675678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.675958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.676234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.676906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:38.677212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.678611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898015-8993-425c-a717-fba313ddc094 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.683011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 420/847] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:38.684847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79454225-f3b9-4d36-befc-477dc1cab96f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.685132 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79454225-f3b9-4d36-befc-477dc1cab96f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.685406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79454225-f3b9-4d36-befc-477dc1cab96f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.686773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 421/848] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:38.687323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79454225-f3b9-4d36-befc-477dc1cab96f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.711567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.712207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.712489 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.712758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.723138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.724559 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:38.739242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:38.740186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:38.741114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:38.741114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:38.741114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:38.741114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:38.741114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 428/849] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:38.741393 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.790241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79454225-f3b9-4d36-befc-477dc1cab96f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.814629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.824781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.825414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.825696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.826001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.826633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.827990 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-da216347-f847-45ee-b2f1-dcfe3bc26725 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:38.836671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:38.837739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:38.838599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 422/850] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:38.839775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.839775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.839775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.849312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db021b-9140-4b6d-a599-6944d535da62 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:38.857251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:38.858481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:38.872464 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:38.883783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:38.884407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:38.884692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:38.884970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:38.885648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:38.885983 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:38.887574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:38.892197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 429/851] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:38.892817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:38.949611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 423/852] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:38.950819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.008862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40f749-fe2e-4f09-81d6-3465d367886f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.047552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.048179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.048459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.048733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.049429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.049763 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.051352 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.052725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.065362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 430/853] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:39.066986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.084853 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258d8b-3321-441c-adb4-db822f09fb2b None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.121488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.122499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.122983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.123478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.124328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.124806 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.126382 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.131310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 424/854] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:39.135437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.150413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.169543 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bce9-6ba9-4954-a97b-c7a19c43c66f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.203394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.203959 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.204230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.204507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.215948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:39.220953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:39.221893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:33:39.222879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:39.227165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 431/855] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:39.228012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.246664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceee9d-4eca-4de6-874d-f03a5dc2a298 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.282395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.283307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.283588 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.283908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.288057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.290814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.292307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:39.304038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:39.305001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:39.305924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:39.310521 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 425/856] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:39.311156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.337706 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb4d0-66d4-46d4-8972-943ec804f6b7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.379258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.379878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.380156 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.380427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.381099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.381401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.382777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.386211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.397970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 432/857] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:39.397970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.425827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56f19-6766-4d4f-a87b-5031201d8d4f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.473169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.473779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.474610 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.474898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.475932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.476239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.478499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.491292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 426/858] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:39.497300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.531814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4e5f-d162-402c-87ac-faabb355aace None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.551978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.576348 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.577238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.577526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.577807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.578620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.578936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.580369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.593745 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 433/859] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:39.599973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.609156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9437-c931-4432-a491-2c61aa50b49c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.658723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.659435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.659718 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.659999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.666385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:39.679630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.681086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:39.695974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:39.696929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:39.697832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:39.697832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:39.697832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:39.697832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:39.697832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 427/860] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:39.704147 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.740961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b9315-ee91-41f2-94f3-bdf3ccf55f66 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.778172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.779042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.779203 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.779578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.780425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=793cd10dcc4344f2be80fd34a82c0d6e) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.780811 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.782335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.793430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 434/861] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/role_assignments?user.id=793cd10dcc4344f2be80fd34a82c0d6e&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:39.824418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ed9-f2fc-474c-9f2c-eccc9b5e66c4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:39.868354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:39.869710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:39.870403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:39.871144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:39.872631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:39.873340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:39.875919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:39.898009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 428/862] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:39.906896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:39.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:40.054577 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e03022-9051-4675-b261-d9feb9472bfd None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:40.093369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:40.094050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:40.094335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:40.094747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:40.103245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:40.104673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:40.120086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:40.121314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:40.122305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 435/863] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:33:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:40.127116 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:40.177289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:40.233141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b761-f5f1-4118-b872-1007d70df720 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:40.268728 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:40.270131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:40.270644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:40.271120 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:40.272101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:40.272638 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:40.274269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:40.292736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 429/864] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jul 27 06:33:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:40.298634 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:40.409316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666118b9-713c-40de-b77e-3387014b9a0a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:40.411686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:40.447100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:40.448112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:40.448599 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:40.449632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:40.522892 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:40.524342 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:40.628983 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 436/865] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:33:40.634410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:40.740710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43827af1-145d-4722-a207-a88d30dd2bfb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:40.743052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:40.775957 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:40.776599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:40.777410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:40.777724 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:40.788028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:40.789466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:40.805429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:40.806355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:40.807265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:40.807265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:40.807265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:40.807265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:40.807265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 430/866] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:40.812705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:40.814864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:40.916944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fa59-9e97-42bb-aff5-d2e14ad53e93 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:40.952254 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:40.952869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:40.953145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:40.953415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:40.954124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:40.954429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:40.955778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38246a-8841-4a00-a696-f633a379edb7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:40.968525 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 437/867] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:40.973695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:41.027653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953393-2fcb-4b74-be64-b30dcf739cfc None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.028102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953393-2fcb-4b74-be64-b30dcf739cfc None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.028455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953393-2fcb-4b74-be64-b30dcf739cfc None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.031174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 438/868] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:41.035396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.035790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.036239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.055389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:41.056582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:41.077298 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff477f-0393-451f-946d-eb187bba0274 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:41.114393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:41.115282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.115708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.116148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.120276 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:41.125978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:41.127815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:41.138211 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 439/869] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:41.142078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:41.143173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:41.144397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 431/870] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:41.149397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:41.163273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:41.183926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:41.275124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227733-6bf1-48de-a246-6f3c475df20f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:41.280733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065c38a-bfa0-4c30-9813-33a8eaa396dc None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:41.311569 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:41.312545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.313041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.313516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.314509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:41.316103 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:41.324795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:41.325662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.325957 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.326211 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.327008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:41.327314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:41.329087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd95f1-4e6a-4442-b474-da735738e6cc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:41.342139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 432/871] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:41 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:41.342710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 440/872] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:41.348862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:41.445653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:41.482977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f8e06-98a0-44ba-afc6-6e2879928419 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:41.509247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0f17-0de3-4c7f-a9d5-ccc8918baf41 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.509787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0f17-0de3-4c7f-a9d5-ccc8918baf41 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.509787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0f17-0de3-4c7f-a9d5-ccc8918baf41 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.511807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 441/873] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:41.517575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2e12-2dcf-469d-8a44-29c068c54f88 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.517869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2e12-2dcf-469d-8a44-29c068c54f88 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.518183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a2e12-2dcf-469d-8a44-29c068c54f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.524879 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:41.525702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.526019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.527546 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.527660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=4a1b7d1e6073474081c39f043ac1ba26) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:41.527919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:41.529587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:41.536040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 433/874] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:41 2022] GET /identity/v3/role_assignments?user.id=4a1b7d1e6073474081c39f043ac1ba26&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:41.540447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-af2a2e12-2dcf-469d-8a44-29c068c54f88 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:41.541666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-af2a2e12-2dcf-469d-8a44-29c068c54f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:41.603967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:41.629080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 442/875] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:41.653756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:41.771047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6d0dc-d027-47dc-be65-6e9a97f75330 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:41.805882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:41.806538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:41.806870 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:41.807148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:41.807840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:41.810223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5f80-561b-4239-bd1c-d47679d22b53 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:41.814630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:41.832801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 434/876] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:41 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:42.086370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a361f9-4b98-49f9-8f85-662aa8bcb7de None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.086867 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a361f9-4b98-49f9-8f85-662aa8bcb7de None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.087255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a361f9-4b98-49f9-8f85-662aa8bcb7de None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.088975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 443/877] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:42.094695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.095105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.095483 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.115069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:42.116399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:42.193187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 435/878] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:42.216439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a361f9-4b98-49f9-8f85-662aa8bcb7de None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:42.321684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a361f9-4b98-49f9-8f85-662aa8bcb7de None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:42.358727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:42.359695 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.360175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.360926 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.373269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:42.378740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:42.379666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:42.380549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:42.383232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 444/879] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:33:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:42.390210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:42.502208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2b34a-bf80-4118-b700-def04bc2220d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:42.525848 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:42.539408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:42.540304 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.540907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.541401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.542349 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:42.542968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:42.544665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:42.554516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 436/880] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:33:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:42.560731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:42.609368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:42.669226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3e6de-723e-4a2f-a8e4-f739c3bdf395 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:42.702804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:42.703434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.703710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.703980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.714890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:42.716402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:42.731347 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:42.732251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:42.733170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:42.733170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:42.733170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:42.733170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:42.733170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 445/881] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:42.738806 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:42.851285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d13c8-d71e-45d0-88d6-f4e2f0f0f12b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:42.887411 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:42.888442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:42.888956 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:42.889472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:42.890418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:42.890944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:42.892522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:42.905527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 437/882] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:42.911284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.019043 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.025479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488045b-1024-49f5-ad6c-a7393054de27 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.061473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.062223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.062509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.062783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.070491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:43.071921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:43.083812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:43.084813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:43.085793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 446/883] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:43.093004 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.141967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.198534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c27a2e-cb02-4aac-a80d-dad5e6259be9 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.235092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.236263 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.236755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.237266 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.238305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:43.238794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:43.240386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa3df3-edd4-4f68-b763-960b57fb2ba7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:43.260029 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 438/884] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:43.267127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.372920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4675c-1344-43fc-b538-d0880d58ca45 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.408664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.409568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.410162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.410638 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.437502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.488920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:43.490698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:43.535878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c693c-dc45-4f9a-bc95-8bf7676fe369 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.536849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c693c-dc45-4f9a-bc95-8bf7676fe369 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.537329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c693c-dc45-4f9a-bc95-8bf7676fe369 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.539156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 439/885] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:43.546909 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.547324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.547693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.567141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:43.569340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:43.610640 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 447/886] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:33:43.611766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.667084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 440/887] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:43.685774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.695438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.726814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.750740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfb0c9-df65-4ae2-accb-db2c78f95da5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.790572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.791685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.792334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.793337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.803943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:43.805628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:43.811820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e1d8-2eb7-4436-925a-9f7813227127 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-a7399ffa-e77e-4f19-9f3b-73c049179170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:43.823080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:43.824221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:43.825258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:43.825258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:43.825258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:43.825258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:43.825258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 448/888] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:43.826153 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c10c-0e7e-4c5a-b8b4-2dc43f1ad9b1 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.826438 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c10c-0e7e-4c5a-b8b4-2dc43f1ad9b1 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.826712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c10c-0e7e-4c5a-b8b4-2dc43f1ad9b1 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.830077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 449/889] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:43.830307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955c10c-0e7e-4c5a-b8b4-2dc43f1ad9b1 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.854547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.855869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.856155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.856434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.857066 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:43.858477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-69c7aa47-2031-4669-8d8b-b883a48e3a33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:43.867897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:43.869130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:43.870187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:43.871584 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 441/890] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:43.872869 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.873155 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.873435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.891803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:43.892989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:43.953096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955c10c-0e7e-4c5a-b8b4-2dc43f1ad9b1 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:43.961199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:43.964991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 442/891] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:43.970423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:43.997406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:43.998054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:43.998340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:43.998616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:43.999303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:43.999610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.000966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.005316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.012738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 450/892] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:44.014809 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.078135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2455257-4f94-47e6-a21c-6e51d2165477 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.113148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.113759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.114071 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.114343 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.114763 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036585f-9b65-4cdb-b7f4-bcf251f3a37b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.115010 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.115312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.116659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9880c46-ee5c-4dcb-90aa-c647152d9bdc None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.121156 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 443/893] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.122318 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7650a9-c83a-451b-bd74-e8a8b86d0b41 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.122599 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7650a9-c83a-451b-bd74-e8a8b86d0b41 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.122871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7650a9-c83a-451b-bd74-e8a8b86d0b41 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.124054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 444/894] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:44.124736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7650a9-c83a-451b-bd74-e8a8b86d0b41 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.153898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.154548 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.154825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.155099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.155721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.157061 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:44.165737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:44.166866 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:44.167893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 451/895] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:44.168708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.200064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.217528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.230838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7650a9-c83a-451b-bd74-e8a8b86d0b41 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.264302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.264932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.265211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.265511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.271713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e955b-5db1-439c-8395-8b0b36fe03e4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.271951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.273376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-c1dbdce6-4c01-4d95-ba2d-8d343d27c8c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:44.284548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:44.285547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:44.286516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 445/896] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:44.287025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.287093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.287605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.305444 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:44.306686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:44.311820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.312435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.312715 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.312991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.313661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.313995 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.315334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.320032 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 452/897] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.320795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.389992 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 446/898] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:44.389992 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.422756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.429740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8df6-85d4-4103-b941-8f993cf545b3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.459987 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.462714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.463345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.463631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.463910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.465096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.465399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.466791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.471405 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 453/899] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.472122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.481305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b252f5-5256-4506-88ee-14eaa7449f6f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.514750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.515401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.515684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.515961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.516708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.517015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.518472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.530309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 447/900] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.531093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.575969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25214-7599-4cd0-b0c3-7dc7643b335c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.609600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.610494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.610786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.611321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.611800 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] RBAC: Adding query filter params (interface=public, service_id=58ef204752bd4ae6b97003469323d2f5, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.612109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.613466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.618841 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 454/901] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/endpoints?service_id=58ef204752bd4ae6b97003469323d2f5&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.619632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.632322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9f9d-2bf0-4615-802e-27b26c4ef5a3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.643876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.668342 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.668988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.669272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.669551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.670209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.671580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:44.680269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:44.681245 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:44.682293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 448/902] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:44.683716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.688003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.727384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6387c-5d60-47a0-84b5-edcd89e57398 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.763198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.763816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.764099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.764375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.765047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.765355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.766731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.771000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 455/903] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.771771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.783554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f69d69-7c40-49c4-89f4-f4e33d97652d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.823017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.824506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.824789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.825928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.825928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] RBAC: Adding query filter params (role.id=a37e92d9517b45bf9dcfe085f74b17cf, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=793cd10dcc4344f2be80fd34a82c0d6e) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:44.826220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.827716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.845985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 449/904] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/role_assignments?user.id=793cd10dcc4344f2be80fd34a82c0d6e&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=a37e92d9517b45bf9dcfe085f74b17cf => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:44.845985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:44.901021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.922932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555abf-cd4e-4b93-a9f3-7b5a08c84ae0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:44.953516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:44.974333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:44.975632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:44.976696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:44.977220 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:44.978471 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:44.981052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:44.994236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 456/905] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services => generated 1268 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:33:45.003030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a90d-fc70-474f-928f-7cf697931541 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:45.045896 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:45.046589 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:45.046873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:45.047152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:45.047852 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:45.048160 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:45.049583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:45.055660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 450/906] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:45.061129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:45.193161 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd189b-a426-4a24-b2fc-aa3c5673a08f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:45.248846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:45.250073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:45.250644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:45.251251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:45.252613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:45.253134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:45.255840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:45.265745 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 457/907] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:45.274003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:45.386240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c71a78-842c-473e-8fcd-9160e3278895 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:45.417067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:45.423374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872355e0-7292-4111-a936-5462177dc94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:45.423998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355e0-7292-4111-a936-5462177dc94e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:45.424278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355e0-7292-4111-a936-5462177dc94e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:45.424543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355e0-7292-4111-a936-5462177dc94e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:45.425431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872355e0-7292-4111-a936-5462177dc94e None admin] RBAC: Adding query filter params (interface=public, service_id=954e80a4dce34c478b446764cc121dd3, region_id=RegionOne) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:45.425744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872355e0-7292-4111-a936-5462177dc94e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:45.427139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872355e0-7292-4111-a936-5462177dc94e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:45.432838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 451/908] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:45 2022] GET /identity/v3/endpoints?service_id=954e80a4dce34c478b446764cc121dd3&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:45.438877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:45.455387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:45.549853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d457-36d0-47a0-8988-43940dd8041c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:45.591105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:45.592198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:45.592701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:45.593175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:45.594075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:45.595755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d294c6-3fc2-499e-b074-a248f36f76fa None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:45.602529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 458/909] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:45 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:33:46.842585 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a0b6a-10e7-46dd-bc3a-883382e53f78 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:46.843209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a0b6a-10e7-46dd-bc3a-883382e53f78 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:46.843209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a0b6a-10e7-46dd-bc3a-883382e53f78 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:46.844756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 452/910] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:46.849968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:46.850381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:46.850575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:46.871587 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:46.872779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:46.911787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ad54a-3668-4d9f-ad9e-f3ba6ac6f7d2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:46.912117 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ad54a-3668-4d9f-ad9e-f3ba6ac6f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:46.912335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ad54a-3668-4d9f-ad9e-f3ba6ac6f7d2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:46.914066 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 453/911] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:46.922757 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:46.923225 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:46.923615 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:46.944172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:46.945443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:46.950330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 459/912] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:46.973322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:46.983820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.038130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 454/913] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:47.060665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.083132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410398d3-8a22-4174-b84d-5c1e8925649c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.122249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.123297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.123784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.124781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.126005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.128369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437107d4-2f78-4f17-b0ad-2be870b65c24 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.149731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 460/914] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:46 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:33:47.151959 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216393-fdce-477e-80c5-4167ce1c6bd5 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.152542 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216393-fdce-477e-80c5-4167ce1c6bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.153349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216393-fdce-477e-80c5-4167ce1c6bd5 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.155848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 461/915] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:47.163870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.164575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.165066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.181749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcfe61-ea44-498b-bb96-4753aede2aed None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.186101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:47.187668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:47.215190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.232143 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.232847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.233131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.233413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.247354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:47.252688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:47.253775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:47.260050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 455/916] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:47.263568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.270931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.281979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 462/917] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:47.305266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.403265 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fe2cf-f437-4ce8-b38d-98602b9c044e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.422720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf16c3-53a2-43ec-9f5f-59ae5cad801f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.445602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.446622 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.447119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.447612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.448563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:47.449079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.450690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.465671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 456/918] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:47.467376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.470362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.471305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.471799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.472277 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.473118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.474694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:47.484523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:47.485849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 06:33:47.486740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:47.493837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 463/919] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:47.496924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.521434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.569934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.600656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e6e2-688a-4293-b893-4d792ab3dddd None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.612662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071f94-8902-4c8d-8959-e7173c443d97 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.649579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.650639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.652177 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.652787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.655197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.656219 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.657903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.658418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.659315 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:47.660630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.665664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.672327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.674049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 464/920] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:47.678983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.679399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-b5e5b421-0511-46c4-939d-bf817997e601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:47.697759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:47.699182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:47.700097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:47.700097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:47.700097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:47.700097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:47.700097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 457/921] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:47.700489 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c84db-b5cb-47bb-bf42-6144288a0657 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.700862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c84db-b5cb-47bb-bf42-6144288a0657 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.700973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c84db-b5cb-47bb-bf42-6144288a0657 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.706086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 458/922] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:47.706086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c84db-b5cb-47bb-bf42-6144288a0657 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.796560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e27172-96f7-4d92-a49a-5ee2fe1bf52c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.803763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.813426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c84db-b5cb-47bb-bf42-6144288a0657 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:47.824987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:47.837350 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.838399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.839283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.840078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.841053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:47.841532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.843628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8f8ca-1924-430d-a63d-c6f8ac25bbf0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.849320 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:47.849968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.850243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.850516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.856322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:47.856322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:47.856322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:47.856637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 465/923] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:47.856637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:47.856637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:47.856637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:47.868548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 459/924] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:47.869283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.879698 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:47.881591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:47.970057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 466/925] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:47.970057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:47.994364 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e114d-53bd-4283-a54e-163f02993fb3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.028572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.029184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.029463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.029736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.030461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.031856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.052059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 460/926] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:48.052988 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.071660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.093124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd5d99-99ee-4154-b5b0-66edb1691b30 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.119487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.144988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.145717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.146159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.146451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.155566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.157049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:48.172141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:48.173189 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:48.174224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:48.175167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fbadb-6144-4cb6-9751-25c68a7cd32a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.175255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 467/927] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:48.179208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.217084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.217732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.218086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.218365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.219014 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.220868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:48.233858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:48.234921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 06:33:48.236075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:48.238608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 461/928] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:48.246692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.303692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a0e3e-2ec9-49af-ae7f-ee94a1d02cbb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.347888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.348466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.348749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.349022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.349825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:48.350533 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.351635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.364690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.373376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 468/929] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:48.379000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.397557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b512e9-f299-48a7-b8c9-734a1fccc2d6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.423442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.449891 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.451256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.451711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.452416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.453299 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:48.453913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.456123 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.468539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 462/930] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:48.470648 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.509352 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19366-32a7-417e-a4e9-d3cfdc251b01 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.551568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.552515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.552795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.553080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.566383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.567827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:48.587256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:48.588231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:48.589664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:48.589664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:48.589664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:48.589664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:48.589664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 469/931] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:48.594162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.600803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529cb1-c611-476c-b72a-e0cbd85fee63 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.641717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.642371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.642650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.642922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.643659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:48.643961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.645496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.656324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 463/932] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:48.658412 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.663746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.723175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:48.750486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0d9c-a288-460c-a5be-09a69574d650 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.788734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.789037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da66a7c-83c0-4a12-818e-8d96f8894fb6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.789890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.790396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.790882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.791770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:48.792870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.794458 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.807154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 470/933] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:48.813230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:48.832317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.833327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.833904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.834570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:48.835356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:48.836780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:48.863138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 464/934] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:33:48.945172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454be29-ebff-41f0-b8f3-d20c5915b3a2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:48.993628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:48.994654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:48.995127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:48.996154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.004106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.005915 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:49.018529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:49.019499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:49.020400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:49.022861 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 471/935] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:49.029053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:49.093060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:49.138326 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f386-4a10-4866-bc82-69368db89880 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:49.172630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:49.173256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.173536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.173811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.174579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:49.174885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.176307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc00d8a-0278-4798-ae14-4dd8ffbb6cff None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.189987 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 465/936] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:49.195898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:49.199241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:49.285324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc4e2-ce05-427a-b568-023214870003 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.286017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc4e2-ce05-427a-b568-023214870003 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.286500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc4e2-ce05-427a-b568-023214870003 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.288433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 466/937] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:49.295664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.296274 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.296772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.303814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf12f9-55b2-4add-9f38-d68a8ef23f71 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:49.316632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:49.318272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:49.346600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:49.347197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.347478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.347752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/793cd10dcc4344f2be80fd34a82c0d6e/roles/a37e92d9517b45bf9dcfe085f74b17cf` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.350633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=793cd10dcc4344f2be80fd34a82c0d6e, role_id=a37e92d9517b45bf9dcfe085f74b17cf)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.352635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.372361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 472/938] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:33:49 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/793cd10dcc4344f2be80fd34a82c0d6e/roles/a37e92d9517b45bf9dcfe085f74b17cf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:33:49.414153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 467/939] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:49.434197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:49.502962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:49.549905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e3c9-7da0-4f61-9055-c97eb37cc587 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:49.587843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:49.588496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.588778 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.589215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.590224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.591567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:49.600938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:49.601985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:49.602865 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:49.606079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 473/940] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:49.609150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:49.663275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:49.725742 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b1339-feea-4ed2-9cf5-85a375728a62 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:49.785463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:49.786502 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.786981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.787459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.788618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:49.789152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.791721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.800563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 468/941] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:49.805986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:49.919382 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a0b2-5f93-4834-959c-a4428db99f05 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:49.946385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:49.955654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:49.956683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:49.958196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:49.958196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:49.958540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:49.959048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:49.960660 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:49.967844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 474/942] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:49.975075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:50.086234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a203bf1-ee85-4a11-939a-625a6ae3f1ae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:50.100699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:50.121778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:50.122417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.122694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.122968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.123723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] RBAC: Adding query filter params (interface=public, service_id=062da8688e8c4bc29896651554129b42, region_id=RegionOne) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:50.124034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:50.125387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a19e4-dfb6-4c4d-8f06-840da2094de2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:50.130924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 469/943] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:49 2022] GET /identity/v3/endpoints?service_id=062da8688e8c4bc29896651554129b42&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:50.137127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:50.236409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e961-f35e-4c90-88e9-83f38fd1eddb None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.237018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e961-f35e-4c90-88e9-83f38fd1eddb None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.237553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e961-f35e-4c90-88e9-83f38fd1eddb None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.239139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 470/944] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:50.245026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.246351 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.246787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2377e-09ce-4723-8960-a8794f06c089 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:50.247141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.266463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:50.268016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:50.285288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:50.285996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.287162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.287654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.288576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:50.290275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:50.305552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 475/945] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:50 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:33:50.347108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 471/946] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:50.371357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:50.434121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:50.481497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefd613-8845-45ef-b651-b38189abe5f6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:50.517411 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:50.518089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.518285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.518514 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.519207 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:50.520619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:50.523233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING keystone.server.flask.common [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 06:33:50.543221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.sql.core [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 06:33:50.543221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 06:33:50.543221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [parameters: {'id': '8cadddcd88a6407abe88d9e80325998a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 06:33:50.543221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86819) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cadddcd88a6407abe88d9e80325998a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cadddcd88a6407abe88d9e80325998a. Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:50.560830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 06:33:50.561704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application yield resource Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise self.value Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application session.commit() Jul 27 06:33:50.563179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self.session.flush() Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application compat.raise_( Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 06:33:50.564274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 06:33:50.565199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:50.567079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [parameters: {'id': '8cadddcd88a6407abe88d9e80325998a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:50.569217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:50.579309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cadddcd88a6407abe88d9e80325998a. Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:50.581524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 476/947] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jul 27 06:33:50 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:50.582864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:50.692401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0323448-4beb-4247-8322-8b46a2e956b5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:50.714355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:50.728423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:50.729068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.729346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.729662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.736961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:50.738417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-7583d632-810a-49cd-a5ed-021f2e0bce94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:50.750518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:50.751493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:50.752858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 472/948] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:50.758063 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:50.862833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e47ef-b8b0-4688-a3ce-3d7c9c22b6d3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:50.873486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:50.898118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:50.898746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.899065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.899346 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.900115 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:50.900419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:50.902201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e758b4c-a685-4c44-bac5-5d7054d52bf7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:50.930042 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 477/949] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:50.976229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a31b8-3b86-43b7-9ed0-acd3142aec91 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.976761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a31b8-3b86-43b7-9ed0-acd3142aec91 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.976761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a31b8-3b86-43b7-9ed0-acd3142aec91 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:50.978533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 473/950] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:50.984969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:50.985753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:50.985753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.015528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:51.017489 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:51.095302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 478/951] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:51.118264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a31b8-3b86-43b7-9ed0-acd3142aec91 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:51.229022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a31b8-3b86-43b7-9ed0-acd3142aec91 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:51.265203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:51.266244 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.266734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.267237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.268065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:51.269624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:51.279019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:51.280040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:51.280911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:51.283666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 474/952] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:51.289850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:51.393982 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:51.396941 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2633-47e1-4461-b65a-90cb6e921353 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:51.436697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:51.437333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.437615 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.437892 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.438602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:51.438968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:51.440381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6570e-b0ce-43a1-a270-06d9754b4255 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:51.445867 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 479/953] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:51.451488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:51.485764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:51.562515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aedce-4a4e-4b31-8e16-9e0f90d0d3c6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:51.598511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:51.599375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.599801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.600252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.601066 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] RBAC: Adding query filter params (type=compute) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:51.601515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:51.603040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b60f0-c159-496f-b1ce-44827438cd9a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:51.608672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 475/954] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:51.621681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfd63-2303-4fb9-8287-02b4d99881f6 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.622158 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfd63-2303-4fb9-8287-02b4d99881f6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.622535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfd63-2303-4fb9-8287-02b4d99881f6 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.625531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 480/955] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:51.630373 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.630781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.631171 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.654022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:51.654022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:51.736873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 476/956] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:51.761172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abfd63-2303-4fb9-8287-02b4d99881f6 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:51.873572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abfd63-2303-4fb9-8287-02b4d99881f6 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:51.909750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:51.910916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:51.911425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:51.911929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:51.922983 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:33:51.928387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:51.929456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:33:51.930477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:51.933720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 481/957] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:33:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:51.939426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:51.967595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.048013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf78cc-beb4-44c3-ad37-c981cbbabc60 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.085421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.086463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.087002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.087504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.088429 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:52.089481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.091210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.100254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 477/958] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:33:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:52.110005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.137082 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.216399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9312d-d58b-4cf3-9192-14905b6ba873 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.256772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.257427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.257710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.258022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.268628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.270121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:52.285487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:52.286535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:52.287831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:52.287831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:52.287831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:52.287831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:52.287831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 482/959] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:52.294304 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.389056 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.401578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0da10-bcf3-40b8-a571-4428463b36f0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.436484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.437089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.437371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.437649 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.438371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:52.438681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.440037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e5948-8b8e-4972-8677-fa4d6ad30541 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.453048 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 478/960] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:52.458935 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.482291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6045d5-5330-4350-a2ce-68abff452a34 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.482553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6045d5-5330-4350-a2ce-68abff452a34 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.482775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6045d5-5330-4350-a2ce-68abff452a34 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.484353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 479/961] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:33:52.489405 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.489678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.489974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.508037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:52.509341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:52.548028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.581301 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b6c35-d888-405f-ba83-0179d1898247 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.598229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 480/962] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:52.616285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.620364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.621221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.621674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.622209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.632261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.634661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:52.651054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:52.652144 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 06:33:52.653145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:52.655619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 483/963] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:52.664658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.715261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.746713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bfbd-1b73-40f4-8885-685c944dc166 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.784477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9f2-eb31-4a92-8545-90303af61cba None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.786858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.787535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.787930 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.789064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.789064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.790639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:52.801316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:52.802701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:52.803752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:52.810294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 481/964] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:52.812436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.831142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.832263 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.832722 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.833638 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.834644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:52.835121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.837248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:52.842013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:52.865848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 484/965] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:52.871593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:52.944404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb2a27-46b9-401e-ac11-a5590fc42c61 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:52.990369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:52.991258 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:52.991556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:52.991835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:52.992931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:52.993252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:52.994915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.003409 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3810b-c03f-4450-9ff7-f10154608d4e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.006018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 482/966] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:53.007672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.019380 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.050653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.051290 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.051567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.051842 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.129497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.134004 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb147-61e5-4bc0-8955-5d18051827ae None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.141923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.143938 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3b4a4-5cdc-4c9a-98e0-34867caf8719 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.173698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.174759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.175267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.175762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.176653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:53.177177 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.178778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425735e6-c270-4c5c-bb2b-44e5807fe444 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.184933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 483/967] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:53.186415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c158-390d-4ba2-8e4b-fa1b05d4949c None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.186940 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c158-390d-4ba2-8e4b-fa1b05d4949c None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.187417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c158-390d-4ba2-8e4b-fa1b05d4949c None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.189096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 484/968] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:53.191689 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25c158-390d-4ba2-8e4b-fa1b05d4949c None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.263495 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 485/969] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:33:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 06:33:53.264539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.264827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.265104 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.270846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.286159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:53.287361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:53.312238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25c158-390d-4ba2-8e4b-fa1b05d4949c None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.347919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.348546 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.348823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.349100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.349802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.351204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.357650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 486/970] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:53.358895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.382789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 485/971] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:33:53.383998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.401666 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.519188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f35351-dc88-4812-b31a-ca228044a70e None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.519640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512238-5742-4e77-ac15-028d371d1d92 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.549344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.565433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.566218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.566498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.566773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.567446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.568863 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.570291 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.571177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.571427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING keystone.server.flask.common [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 06:33:53.571886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.572370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.584955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.586759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.587580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.sql.core [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 06:33:53.587580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 06:33:53.587580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [parameters: {'user_id': 'a1a96960128a4610a4b0021e99b0a2e0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 06:33:53.587580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86820) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:53.595505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:53.596461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application yield resource Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application next(self.gen) Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise self.value Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application session.commit() Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 06:33:53.597446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.session.flush() Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application compat.raise_( Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 06:33:53.598538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 06:33:53.599459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application conn.query(q) Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application result.read() Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 06:33:53.600314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a1a96960128a4610a4b0021e99b0a2e0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:53.601195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 06:33:53.602069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.602991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 486/972] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jul 27 06:33:53 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:53.604585 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:33:53.612054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:53.613742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:53.614940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:33:53.614940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:33:53.614940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:33:53.614940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:53.614940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 487/973] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:53.630166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.724119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.739883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78ad47-191a-4d0e-8d29-9d1df291c675 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.748213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e89bb-4514-4f46-b1e5-70b14158f57f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.782644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.783652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.783983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.784213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.785573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.786235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.786523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.786795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.787489 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:53.787804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.789159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386d228-31b5-40a4-aa6b-18d99a68f52c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.792410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.794108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.805732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 488/974] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:53.805732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271def1-720c-4676-b079-95014efeb173 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.805732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271def1-720c-4676-b079-95014efeb173 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.805732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271def1-720c-4676-b079-95014efeb173 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.805732 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 489/975] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:53.809719 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271def1-720c-4676-b079-95014efeb173 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d88b73b1-fab2-4f35-8ba0-df05ab967b7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:33:53.812815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:53.813729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 06:33:53.814714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:53.818585 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 487/976] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:53.819857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.820150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.820435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.839531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:53.841058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:53.884259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.917798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 488/977] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:53.919158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:53.931391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271def1-720c-4676-b079-95014efeb173 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:53.970246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:53.971247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:53.971727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:53.972192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:53.975954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:53.982199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:53.983631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:33:53.996380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:53.997336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:53.998295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 490/978] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:53.998825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:54.029549 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4daa1-b477-4e76-9442-92f34dfcb0b1 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:54.069615 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:54.070355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:54.070850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:54.071283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:54.072506 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:54.072944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:54.074825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:54.107780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6626de-c445-4b7f-a849-e6341f222cd9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:54.121983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 489/979] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jul 27 06:33:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:54.121983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:54.153291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:54.160041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:54.160686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:54.160974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:54.161255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:54.161987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:54.163399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:54.185971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 491/980] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:53 2022] POST /identity/v3/services => generated 242 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:54.244203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036717c-4491-44d7-aa4d-1ac90c2d9703 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:54.294384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:54.295030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:54.295310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:54.295583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:54.296361 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:54.296667 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:54.298164 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee25c5d-b40e-4c65-8f38-ad8635189009 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:54.312616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 490/981] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:54.318566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:54.383566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:54.433480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6118627-1068-4184-ade6-810c768365b7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:54.468384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:54.469002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:54.469279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:54.469551 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:54.470361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] RBAC: Adding query filter params (role.id=a37e92d9517b45bf9dcfe085f74b17cf, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=793cd10dcc4344f2be80fd34a82c0d6e) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:54.470669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:54.472092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ad20f-9d91-4498-9022-cb30eeec62ed None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:54.477933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 492/982] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:33:54 2022] GET /identity/v3/role_assignments?user.id=793cd10dcc4344f2be80fd34a82c0d6e&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=a37e92d9517b45bf9dcfe085f74b17cf => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:55.489129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820bb61-f3c2-4420-a85f-cca37815b598 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:55.489654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820bb61-f3c2-4420-a85f-cca37815b598 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:55.489762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820bb61-f3c2-4420-a85f-cca37815b598 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:55.491228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 491/983] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:55.496870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:55.497248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:55.497434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:55.515380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:55.516572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:55.586179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 493/984] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:55.609670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820bb61-f3c2-4420-a85f-cca37815b598 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:55.708688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:55.723195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820bb61-f3c2-4420-a85f-cca37815b598 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:55.758282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:55.758839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:55.759117 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:55.759388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:55.760002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:55.761399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:55.770270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:55.771256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:55.777705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 492/985] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:55.780047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:55.829678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:55.881106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6633c56-eb76-48fb-8129-d79c1d90d065 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:55.914988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:55.915613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:55.915896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:55.916173 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:55.916988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:55.917304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:55.918688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:55.928233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 494/986] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:55.929875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.050353 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18db17-6dda-4a65-82c8-15aba4d99344 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.085209 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.085831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.086139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.086420 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.087095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:56.087403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.088832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.098601 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 493/987] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:56.100141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.203206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977d500-8ef3-437a-8442-1e3841ab3da6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.217897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.240183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.240805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.241088 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.241365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.242156 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] RBAC: Adding query filter params (interface=admin, service_id=062da8688e8c4bc29896651554129b42, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:56.242475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.243826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e360d-5f95-4213-a8ce-c93220bc4a7a None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.253449 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 495/988] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/endpoints?service_id=062da8688e8c4bc29896651554129b42&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:56.254876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.275308 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.299885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00626-8119-4484-9d08-8fff76edeb1a None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.300205 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00626-8119-4484-9d08-8fff76edeb1a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.300498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00626-8119-4484-9d08-8fff76edeb1a None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.301924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 496/989] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:56.306531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.306817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.307089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.325804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:56.327041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:56.365076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c4571-4253-4d86-9ba8-4512b5d78f75 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.397727 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 497/990] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:56.399630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.400246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.400523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.400843 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.401772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.403257 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3594d-199b-483c-a546-0eaa1b495723 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.413982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 494/991] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:33:56.420823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.519247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f2e6-e4f5-4337-9ac0-961372148cbd None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.519793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f2e6-e4f5-4337-9ac0-961372148cbd None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.519793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f2e6-e4f5-4337-9ac0-961372148cbd None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.521170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 495/992] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:56.527212 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244818d-aa47-40bd-9230-939802eb053f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.527502 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244818d-aa47-40bd-9230-939802eb053f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.527784 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244818d-aa47-40bd-9230-939802eb053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.530804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9649ff-6ddc-4cbc-a98a-482ce64b217d None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.541386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.547250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-7244818d-aa47-40bd-9230-939802eb053f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:56.548488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-7244818d-aa47-40bd-9230-939802eb053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:56.570921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.571494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.572105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.572105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.572658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.574081 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:56.583940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:56.584993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:56.586044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 498/993] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:56.591832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.627582 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 496/994] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:56.649689 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244818d-aa47-40bd-9230-939802eb053f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.665999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.707927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d040ac-ed2c-449b-8fa2-74ee48197a48 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.742953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.743579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.743863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.744142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.744829 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:56.745665 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.747079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.751975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 499/995] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:56.757388 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.763952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244818d-aa47-40bd-9230-939802eb053f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.795658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.800317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.801227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.801516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.801794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] PATH_INFO: `/v3/services/image` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.802452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.803811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:56.813544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:56.814811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:56.815671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 497/996] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:56.820473 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.869091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8387f-fb98-4e5c-8890-35802877e105 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.903200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:56.907846 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.908478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.908759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.909031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.909700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:56.910034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.911390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.921248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 500/997] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:56.922734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:56.928888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f01923-e8ba-4fe3-a6c6-28510d7c76f6 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:56.966389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:56.967012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:56.967292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:56.967567 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:56.968244 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] RBAC: Adding query filter params (name=image) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:56.968549 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:56.969985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:56.980229 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 498/998] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:56.982184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:57.031008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:57.034419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f3ca5-286b-4b60-b43e-694d38946c79 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:57.072584 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:57.073789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:57.074210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:57.074486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:57.075357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] RBAC: Adding query filter params (interface=public, service_id=8597a8d0363b401695e1afa6a5f2fd20, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:57.075673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:57.077036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:57.087249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 501/999] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/endpoints?service_id=8597a8d0363b401695e1afa6a5f2fd20&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:57.088260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:57.096211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a86a8-263b-4fbe-b4c9-cf7ca852be3d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:57.130815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:57.131348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:57.131633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:57.131918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:57.132603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] RBAC: Adding query filter params (type=image) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:57.132914 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:57.134390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa560a-f54b-401a-903d-b9889a245e2f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:57.140070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 499/1000] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:33:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:57.207699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e02fd-6a86-4a81-94c5-4beb5d614c99 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:57.244140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:57.244775 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:57.245602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:57.246124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:57.246893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:57.248285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28241364-5503-4841-b2ce-9bfe76bd9522 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:57.262255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 502/1001] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:33:57 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:33:58.462686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1ef6-3c63-4f20-aaf0-78120eeacde0 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:58.463211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1ef6-3c63-4f20-aaf0-78120eeacde0 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:58.463289 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a1ef6-3c63-4f20-aaf0-78120eeacde0 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:58.464962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 500/1002] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:58.469838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:58.470265 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:58.470432 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:58.489176 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:58.490391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:58.513494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:58.558114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 503/1003] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:33:58.580337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1ef6-3c63-4f20-aaf0-78120eeacde0 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:58.583237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:58.685052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a1ef6-3c63-4f20-aaf0-78120eeacde0 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:58.719608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:58.720161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:58.720439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:58.720712 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:58.721332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:58.722844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:58.732522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:58.733542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:33:58.740097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 501/1004] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jul 27 06:33:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:33:58.742380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:58.841933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f42ef1-61d3-4f8b-98f2-e7098c918282 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:58.876031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:58.876939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:58.877501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:58.877973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:58.878821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:58.879303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:58.880812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:58.886349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 504/1005] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:58.891556 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:58.957880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:58.994600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79512f5c-6125-4614-b205-f8943c9ba6db None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.033348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.034302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.034808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.035259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.036107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:59.036600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.038164 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d179-a1fb-488a-b6e2-36b40183287c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.043982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 502/1006] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jul 27 06:33:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:33:59.049363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:59.075539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:59.137416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbb14-f050-4e03-bb69-4ba3e7b60d3b None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.137738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbb14-f050-4e03-bb69-4ba3e7b60d3b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.138013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbb14-f050-4e03-bb69-4ba3e7b60d3b None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.139526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 503/1007] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:59.144256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b045-37d2-492a-b544-814a81345550 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.144545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b045-37d2-492a-b544-814a81345550 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.144823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11b045-37d2-492a-b544-814a81345550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.162481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-df11b045-37d2-492a-b544-814a81345550 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:59.163665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-df11b045-37d2-492a-b544-814a81345550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:59.169396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dddaf-8d0b-4275-8306-cdc54eac929c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.206570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.207194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.207479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.208166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.208893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.210643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a817e-2e54-4a4e-9518-124bfc35b481 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.237975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 505/1008] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:59.250855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 504/1009] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:59.269875 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1056-154f-4ee0-a9ed-8200efd26c04 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.270258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1056-154f-4ee0-a9ed-8200efd26c04 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.270542 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1056-154f-4ee0-a9ed-8200efd26c04 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.272109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 506/1010] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:33:59.273834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11b045-37d2-492a-b544-814a81345550 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:59.278125 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.278412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.278691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.301511 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:33:59.303142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:33:59.339516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:59.396141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 507/1011] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:33:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:33:59.396336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11b045-37d2-492a-b544-814a81345550 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.420520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:59.435094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.435785 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.436152 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.436267 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:59.436431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.437130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.438827 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.456246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 505/1012] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:33:59 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:33:59.541151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef4bd2-aad1-4e89-9f60-5a2a0df3543c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.585786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.586473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.586760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.587075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.587716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.589973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:33:59.599708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:33:59.601009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:33:59.602231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 508/1013] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:33:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:33:59.610891 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:59.718006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5495-063b-4259-91d6-5b6cebf04f27 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.737113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:59.758118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.758746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.759034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.759312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.759998 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] RBAC: Adding query filter params (name=compute) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:59.760307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.761694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.766781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 506/1014] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:59.772578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:33:59.847180 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:33:59.878628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bec391-b7dc-4b34-88ec-fa97d1b04351 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:33:59.913190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:33:59.913813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:33:59.914135 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:33:59.914418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:33:59.915095 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:33:59.915409 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:33:59.916822 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b655e2-5b33-491b-809a-11bc20133bc9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:33:59.926649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 509/1015] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jul 27 06:33:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:33:59.927579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:00.031704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0836d-2386-46e5-b0a5-05cb4dee524f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:00.064652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:00.065270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:00.065568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:00.065838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:00.066581 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:00.067934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1588a9-a185-4114-b4a1-f0fe6706ab4c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:00.093967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 507/1016] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:33:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:34:01.274366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927bcb3-eac7-4a03-a52d-94b6cff347a3 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.274942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927bcb3-eac7-4a03-a52d-94b6cff347a3 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.274942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927bcb3-eac7-4a03-a52d-94b6cff347a3 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.276721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 510/1017] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:01.282256 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b04c-6b1b-4eb9-8f9e-d8e419d80b63 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.282799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b04c-6b1b-4eb9-8f9e-d8e419d80b63 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.282799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28b04c-6b1b-4eb9-8f9e-d8e419d80b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.301339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-cb28b04c-6b1b-4eb9-8f9e-d8e419d80b63 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:01.302553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-cb28b04c-6b1b-4eb9-8f9e-d8e419d80b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:01.333020 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:01.382092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 508/1018] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:01.404330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927bcb3-eac7-4a03-a52d-94b6cff347a3 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:01.508598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927bcb3-eac7-4a03-a52d-94b6cff347a3 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:01.523661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2615e-4ab6-4b70-985d-a38fb8c88c43 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.523948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2615e-4ab6-4b70-985d-a38fb8c88c43 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.524226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2615e-4ab6-4b70-985d-a38fb8c88c43 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.525778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 509/1019] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:34:01.533799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.534121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.534401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.543540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:01.544184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.544465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.544740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.545883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:01.547386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:01.550152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 511/1020] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:01.552819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:01.554158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:01.555048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:01.560011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:01.627538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 510/1021] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:01.650725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:01.663521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352617-c40c-41d0-abda-2dfdc39844cb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:01.700200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:01.700828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.701111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.701388 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.702109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:01.703522 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:01.746099 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 512/1022] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:34:01 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:34:01.767012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:01.778187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a11f-c9ef-45ef-b5d9-d7b3125c4f09 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:01.816476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:01.817240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.817521 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.817838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] PATH_INFO: `/v3/services/image` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.818504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:01.819936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:01.830055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:01.831165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:01.837028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 511/1023] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 06:34:01 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:01.838952 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:01.899780 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:01.945415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1d17f-b7fd-426e-9bb8-1e90101ee18f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:01.980447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:01.981078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:01.981362 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:01.981640 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:01.982350 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] RBAC: Adding query filter params (name=image) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:01.982660 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:01.984014 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:01.993714 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 513/1024] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:01.996717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.102908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff06-d499-4ea7-a110-6bdfc392cd5d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.137760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.138410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.138690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.138965 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.139637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] RBAC: Adding query filter params (type=image) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:02.139939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:02.141287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01d2d4-cca2-4700-83b2-0ab345052c9b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:02.146463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 512/1025] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:02.152566 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.193350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a062639-2b4c-4234-bc1e-2485e4b26f7e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.193671 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a062639-2b4c-4234-bc1e-2485e4b26f7e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.193913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a062639-2b4c-4234-bc1e-2485e4b26f7e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.195441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 513/1026] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:34:02.200493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.200779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.201062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.218899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:02.220070 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:02.236037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:02.265610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f13c-4e69-49bf-af97-9683c49ec235 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.286463 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:02.302054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 514/1027] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:02.309338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.309987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.310273 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.310552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.311312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] RBAC: Adding query filter params (interface=public, service_id=f7fe947fd4444df68b6bf77e3ae3cced, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:02.311626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:02.312989 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:02.322844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 514/1028] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/endpoints?service_id=f7fe947fd4444df68b6bf77e3ae3cced&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:02.324100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.324802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.431051 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc134eb6-f4a1-4a7a-a810-5f159741bb30 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.435837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1bd-3880-4d39-a621-691b2a561951 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.469431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.470113 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.470400 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.470681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.471340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:02.472186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.472701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:02.473157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.473611 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.474062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.480443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 515/1029] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:34:02.496810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:02.501992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:02.503023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:02.503907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:02.506349 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 515/1030] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:02.509387 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.527559 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:02.621006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f83399-1a5a-499f-b1e4-dce5b0ec8726 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.661138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.661767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.662088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.662368 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.663089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:02.663394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:02.664759 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:02.673935 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 516/1031] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:02.679799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.730082 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:02.785623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb91c-4aae-4045-be6f-a3a153f013eb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.819915 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.820556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.820838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.821111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.831858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:02.833292 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:02.848504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:02.849439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:02.850756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:02.850756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:02.850756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:02.850756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:02.851304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 516/1032] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:02.856267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:02.937673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:02.962903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d83201-b035-4ba2-8a79-ff7a096a4652 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:02.997668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:02.998240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:02.998525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:02.998801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:02.999493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:02.999799 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:03.001157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:03.017566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 517/1033] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:03.019053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.114739 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:03.121843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4953-51d7-49d5-b8b4-56921fc683c5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:03.156048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:03.156775 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.157127 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.157470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.165537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:03.167895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:34:03.179297 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:03.180428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:03.181624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:03.187906 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 517/1034] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:03.189375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.293189 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814a83-d05b-4658-a907-fa2839979d7c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:03.328128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:03.328737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.329012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.329288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.330105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:03.330417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:03.331831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:03.334450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:03.354009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 518/1035] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:03.356853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.459069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeabd3f-58fb-4339-a92a-f4fbcaa8273c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:03.492331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:03.492956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.493241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.493518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.518748 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:03.563774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:03.565131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67626aae-9f60-44e2-ae24-d08bbea478c2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:03.688064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 518/1036] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:03.689242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.788240 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9a9b9-139d-4f6f-87dd-c9532152ace4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:03.821957 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:03.822524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.823426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.823705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.827971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c22b-2ff7-4831-86ff-2e5ed9baedc5 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.828260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c22b-2ff7-4831-86ff-2e5ed9baedc5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.828535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c22b-2ff7-4831-86ff-2e5ed9baedc5 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.833545 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 519/1037] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:34:03.834664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:03.834944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:03.835214 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:03.837908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:03.839846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:03.853534 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:03.854413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:03.855425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:03.859073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:03.859073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:03.859073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:03.859073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:03.859300 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:03.859374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 519/1038] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:03.861466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.908629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:03.911149 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:03.928509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 520/1039] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:03.953000 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:03.969855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c521d-301e-485b-8460-eba2c758d590 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.007989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.009278 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.010051 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.010519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.012002 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:04.012784 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.014871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.031283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 520/1040] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:04.036568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.055686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4e737-2d9c-4a13-be7d-c296fc0ed024 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.094885 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.095529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.095810 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.096080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.097213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.098654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.100507 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 521/1041] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:04.106343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.151304 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc5406-94be-439a-84e6-16b4b521dad8 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.159072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:04.167270 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:04.192382 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.193334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.193622 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.193926 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.202147 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.203600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.214241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a1f74-434b-490c-ac2f-75eb0b7ea943 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:34:04.215150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:04.216040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:04.217007 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:04.221243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 521/1042] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:04.221860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.253839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.261108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.261108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.261108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.261108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.261751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.285854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 522/1043] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:34:04 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:34:04.339539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bcd1c-5d72-4ff5-ab92-c81742f9ff0e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.379328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.379885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.380171 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.380450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.381313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:04.381628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.383151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7e2a0-7b4c-4d31-b459-49c01a34c463 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.401989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 522/1044] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:04.404335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.516942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613beb2-9b57-4fa3-9d3b-c52d21cb8157 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.528243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:04.535490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf8e8-83dd-43a9-8aec-1cc60439fb7e None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.535761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf8e8-83dd-43a9-8aec-1cc60439fb7e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.536043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf8e8-83dd-43a9-8aec-1cc60439fb7e None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.537642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 523/1045] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:04.543518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.543805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.544087 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.554377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.554995 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.555276 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.555548 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.556330 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] RBAC: Adding query filter params (role.id=a37e92d9517b45bf9dcfe085f74b17cf, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:04.556637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.558414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.563591 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:04.569969 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 523/1046] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=a37e92d9517b45bf9dcfe085f74b17cf => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:04.570176 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:04.593453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:04.648050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 524/1047] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:04.672334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.777466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0692699-bea1-4fbd-9503-a08405098f36 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.810914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.811552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.811829 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.812100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] PATH_INFO: `/v3/services/image` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.812723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.814100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:04.824806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:04.825917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:04.826876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 524/1048] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:04.833764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:04.933359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbe5b-64ad-42b5-9510-1cead4725cb4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:04.966449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:04.967066 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:04.967347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:04.967616 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:04.968285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] RBAC: Adding query filter params (name=image) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:04.968588 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:04.969962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:04.980237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 525/1049] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:04.981309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:05.023140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:05.085041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f006f9-1a2e-4f41-86d4-67b45d7fe6ae None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:05.120886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:05.121499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:05.121773 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:05.122070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:05.122742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] RBAC: Adding query filter params (type=image) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:05.123046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:05.124390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd02ced-615a-4c69-bc96-de956898cc1f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:05.133996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 525/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:05.135231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:05.157079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:05.241144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253ef35-cab7-4e3d-a26a-92dc79aed642 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:05.274366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:05.274975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:05.275253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:05.275526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:05.276275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:05.277704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a10e6f-4755-4a59-a6f2-5bf3115fcddc None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:05.305977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 526/1051] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:34:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:34:06.368309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf8ee-07d6-4040-a807-4e2cbb831a0f None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.368921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf8ee-07d6-4040-a807-4e2cbb831a0f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.368921 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf8ee-07d6-4040-a807-4e2cbb831a0f None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.371410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 526/1052] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:06.376679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65e7a-cf9e-4bd6-b6f3-02a47f6758b4 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.377138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65e7a-cf9e-4bd6-b6f3-02a47f6758b4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.377228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65e7a-cf9e-4bd6-b6f3-02a47f6758b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.396360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-8db65e7a-cf9e-4bd6-b6f3-02a47f6758b4 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:06.397535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8db65e7a-cf9e-4bd6-b6f3-02a47f6758b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:06.481380 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 527/1053] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:06.507627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf8ee-07d6-4040-a807-4e2cbb831a0f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:06.570882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af708-a2bb-4d89-aacd-b35cf7ad2c44 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.571177 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af708-a2bb-4d89-aacd-b35cf7ad2c44 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.571461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af708-a2bb-4d89-aacd-b35cf7ad2c44 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.573128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 528/1054] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:06.578030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.578320 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.578597 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.585732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:06.598836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:06.600029 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:06.623493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf8ee-07d6-4040-a807-4e2cbb831a0f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:06.625155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:06.663761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:06.664406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.664688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.664964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.676476 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:06.679357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 529/1055] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:06.681390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:06.682466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:06.688552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 527/1056] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:06.690913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:06.701965 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:06.798788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72856e-6bbb-4f1f-8a26-52ac1671fd91 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:06.805101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65382-a30d-43c5-9b46-91c984c96e50 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:06.830064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:06.835910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:06.836529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.836809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.837082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.837793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:06.838130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:06.839328 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:06.839624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:06.839972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:06.840255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:06.840531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:06.852693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 530/1057] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:06.854535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:06.856830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:06.863686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:06.864664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:06.865573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:06.867985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 528/1058] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:34:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:06.873231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:06.898755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:06.980112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f802c-5123-473f-9563-1cc35cf866e0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:06.986082 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857c0d-f677-4a37-b34a-f4517e70c674 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.015251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.015898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.016177 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.016454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] PATH_INFO: `/v3/users/eef62dd6b34042c88ed02d179d7a65f9` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.020832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.021453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.021738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.022047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.022766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:07.023076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.024432 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.031314 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] RBAC: Authorizing `identity:get_user(user_id=eef62dd6b34042c88ed02d179d7a65f9)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.032741 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.033616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 529/1059] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:34:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:07.039152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.039737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 531/1060] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:06 2022] GET /identity/v3/users/eef62dd6b34042c88ed02d179d7a65f9 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:07.040490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.098154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.128527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.145142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820306e1-0d56-4bdf-8967-081996f63190 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.151608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddec93f-2eb6-49c4-9fa2-ea1bbb1ec474 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.179870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.180518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.180799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.181075 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.189509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.190760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.191216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.191915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.192368 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.193799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:07.210345 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:07.211488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:07.212428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:07.212428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:07.212428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:07.212428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:07.212428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 530/1061] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:07.218733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.277786 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.279337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bdae4-5b87-4a3b-91f7-ba848a839162 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.325728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74d165-91f3-4358-b470-0e226d3fe688 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.332979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.359955 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.360830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.361114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.361389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.362113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:07.362428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.363784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23df738-9413-484d-9025-45d9ad539455 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.366048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.383680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 531/1062] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:07.385517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b87f3-35d9-4d52-b048-49aaa73d0c94 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.386240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b87f3-35d9-4d52-b048-49aaa73d0c94 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.386240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b87f3-35d9-4d52-b048-49aaa73d0c94 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.388174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 532/1063] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:07.398167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 532/1064] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:07.398728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b87f3-35d9-4d52-b048-49aaa73d0c94 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.399504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.399792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.400065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.419116 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:07.420227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:07.486279 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 533/1065] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:07.486907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.507064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b87f3-35d9-4d52-b048-49aaa73d0c94 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.542876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.543501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.543784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.544056 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.550879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.552286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:34:07.564152 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:07.565108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 533/1066] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:07.566131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.571052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.590542 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39093f-e760-4957-92f6-af415526cbb7 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.624577 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.625211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.625491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.625764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.626600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.626921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.628449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.635140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 534/1067] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:07.635140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.672370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbeb83-c081-4de9-a049-ad395a31ba36 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.705768 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.706414 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.706701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.707009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.734288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecc35d-7d64-4fe7-aca7-bceb61e19d17 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.767949 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.768563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.768841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.769115 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.769895 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:07.770230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.771690 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.772783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.774224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.788950 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 535/1068] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:07.789557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.800157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.831344 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:07.874958 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 534/1069] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:07.875788 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.892264 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21ae3-130e-4324-97f2-94b0059de1ca None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:07.926269 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:07.926810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:07.927086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:07.927361 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:07.928137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=94342ea7362f4a9b901f7f8071ae46b7, user.id=eef62dd6b34042c88ed02d179d7a65f9) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:07.928445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:07.929850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:07.935515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 536/1070] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/role_assignments?user.id=eef62dd6b34042c88ed02d179d7a65f9&scope.project.id=94342ea7362f4a9b901f7f8071ae46b7&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:07.936376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:07.982110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c396-fbfe-4405-b8d2-dcc7d7baa314 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.012931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:08.023543 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.024205 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.024490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.024770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.037008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.038504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.047282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fbe0-2980-490a-8370-733744f65570 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:08.054581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:08.055606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:08.056653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:08.056653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:08.056653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:08.056653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:08.056653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 535/1071] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:08.061733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:08.075032 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:08.092063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.092774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.093059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.093327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.106003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.107499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:08.122634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:08.123753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 537/1072] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:08.129683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:08.187184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d10900-ea71-4df3-9d73-7b6efaa74b9b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.230855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.231605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.231890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.232168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.232725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccafc-68cc-41fc-8c45-05bb2064615f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.232895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:08.233167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.234731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.251597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 536/1073] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:08.257255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:08.257803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:08.270633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.271413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.271701 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.271982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.272782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:08.273100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.274516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.287905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 538/1074] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:08.339884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:08.380667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e168a7-6507-4098-b023-1797d73c8fa5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.419240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.419926 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.420171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.420446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.428188 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.429663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:34:08.442539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:08.443480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:08.444385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 537/1075] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:08.449488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:08.561208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a634-a6ba-4497-a31b-f32507731e6d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.602022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.602903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.603186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.603466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.605183 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:08.605498 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.607012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24df719-88f5-4a43-8416-67fa3673229b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.619090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 539/1076] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:08.625453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:08.746506 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ab8da-ad46-4f58-a96a-8a3eb88c9037 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:08.778091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:08.788993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:08.789668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:08.789959 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:08.790259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/a37e92d9517b45bf9dcfe085f74b17cf` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:08.793269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=b426ebd709e942819f19926f2e3668d2, role_id=a37e92d9517b45bf9dcfe085f74b17cf)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:08.794855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53b493-1467-4427-aa64-d153ddd576e6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:08.812527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 538/1077] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:34:08 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/b426ebd709e942819f19926f2e3668d2/roles/a37e92d9517b45bf9dcfe085f74b17cf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:34:09.977862 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbedde-6868-4759-a88f-c563e3fcc939 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:09.978435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbedde-6868-4759-a88f-c563e3fcc939 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:09.978435 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbedde-6868-4759-a88f-c563e3fcc939 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:09.979739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 540/1078] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:09.985317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:09.985747 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:09.985878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.003986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:10.005156 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:10.068419 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 539/1079] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:10.090451 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbedde-6868-4759-a88f-c563e3fcc939 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.136489 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:10.190873 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbedde-6868-4759-a88f-c563e3fcc939 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:10.224335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:10.224975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.225259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.225543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.235951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:10.240813 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:10.241906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:10.242831 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 541/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:10.248434 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.343899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a12ca-7342-40aa-93e3-6b352c6d475d None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:10.361450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:10.380155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:10.380770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.381049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.381323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.382055 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:10.382361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:10.383710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:10.392196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 540/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:10.397742 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.497620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc3dd-85a0-4413-9129-03fef90d909a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:10.499892 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:10.532292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:10.532923 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.533210 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.533484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] PATH_INFO: `/v3/users/eef62dd6b34042c88ed02d179d7a65f9` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.534650 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] RBAC: Authorizing `identity:get_user(user_id=eef62dd6b34042c88ed02d179d7a65f9)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:10.536078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:10.537667 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 542/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:10 2022] GET /identity/v3/users/eef62dd6b34042c88ed02d179d7a65f9 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:10.544100 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.642932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cdd52-76aa-498c-a5cc-c0d46c694288 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:10.678412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:10.679051 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.679333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.679605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.680760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:10.682972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b33f8-7beb-439c-bbb5-529e55c11287 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:10.684651 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 541/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:10 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:10.690655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.788611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a91a-1a08-4ffe-8ec3-49e31b3587a2 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:10.806338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ee598-6758-44ac-b422-54e43ffc4459 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.806631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ee598-6758-44ac-b422-54e43ffc4459 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.807074 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ee598-6758-44ac-b422-54e43ffc4459 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.808442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 542/1084] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:34:10.813285 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.813570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.813849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.821062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:10.821719 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:10.822021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:10.822293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7/users/eef62dd6b34042c88ed02d179d7a65f9/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:10.824219 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] RBAC: Authorizing `identity:create_grant(project_id=94342ea7362f4a9b901f7f8071ae46b7, user_id=eef62dd6b34042c88ed02d179d7a65f9, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:10.825637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:10.832638 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:10.833835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:10.839121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 543/1085] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:34:10 2022] PUT /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7/users/eef62dd6b34042c88ed02d179d7a65f9/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:34:10.881151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:10.907110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 543/1086] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:10.930065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:10.964399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:11.036893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56470fa-bc86-41c8-8fbf-65bd684c5e11 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.069847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.070449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.070733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.071011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.081427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:11.086855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:11.088027 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:11.088893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 544/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jul 27 06:34:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:11.094272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.191129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc519e-5268-4a6e-9953-c03bbe742714 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.223612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.224224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.224501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.224772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.225475 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:11.225779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:11.227154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:11.236260 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 544/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:11.241805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.331108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:11.339898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197001ad-1cc4-478c-af78-e93633545880 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.373219 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.373855 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.374158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.374436 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.384681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:11.386385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:11.399967 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:11.401035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:11.401973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:11.401973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:11.401973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:11.401973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:11.401973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 545/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:11.406858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.449961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:11.507967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56247580-bd31-4cca-a036-89e5037ddff6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.542710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.543319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.543602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.543880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.544567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:11.544880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:11.546244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:11.558342 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 545/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:11.563899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.669881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfd2b9-7542-4b32-8cae-9eebe3d38cb4 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.702276 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.702904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.703180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.703452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.710538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:11.711968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:34:11.723485 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:11.724533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:11.725495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 546/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:11.730997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.824052 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:11.827264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fe1fe-1b64-449a-97cf-95166ec1c2d2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:11.861018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:11.861691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:11.862002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:11.862283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:11.863065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:11.863376 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:11.864784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:11.881377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 546/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:11.887729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:11.891664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:11.989698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb00b6-f64f-4f73-b558-172dc6d6051b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.022554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.023166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.023466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.023723 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.090109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.091597 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.094169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:12.188544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 547/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:12.194021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.294035 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24760a1b-25a7-4ac0-9384-5d6025577639 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.327337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.327975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.328261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.328542 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.338998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.340418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:12.356720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:12.357825 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:12.358807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:34:12.358807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:12.358807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:34:12.358807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:12.358807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 547/1094] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:12.363952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.460776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804e271-5b36-46d2-8eaf-73ae16ef2da7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.463815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:12.495890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.496508 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.496793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.497074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.497760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:12.498107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.499483 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.510488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 548/1095] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:12.516087 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.529202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:12.615165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd8134-6506-4f79-8a32-18dc63660f45 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.647113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.647738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.648014 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.648283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.657421 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.658964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:34:12.671486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:12.672570 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:12.678200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 548/1096] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:12.678713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.775166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadbf5c-c387-4cf8-8900-65c1e08e8a13 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.808282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.808888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.809166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.809440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.810215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:12.810523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.811931 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d0998-d042-481f-91a8-1bba42841b06 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.823515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 549/1097] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:34:12.829548 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.831388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3681-1ea5-4e4b-8880-f730b7f0c49d None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.831677 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3681-1ea5-4e4b-8880-f730b7f0c49d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.831955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3681-1ea5-4e4b-8880-f730b7f0c49d None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.833237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 550/1098] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:34:12.838142 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a24698-72e8-410e-b688-8e444adde63a None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.838999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a24698-72e8-410e-b688-8e444adde63a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.839277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a24698-72e8-410e-b688-8e444adde63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.856927 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-01a24698-72e8-410e-b688-8e444adde63a None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:12.858133 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-01a24698-72e8-410e-b688-8e444adde63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:12.876428 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:12.899336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:12.925982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 551/1099] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:12.931180 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e187b-e9fe-4a12-9f93-155fbea55d5c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:12.948858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a24698-72e8-410e-b688-8e444adde63a None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:12.966149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:12.966766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:12.967049 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:12.967322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:12.968096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] RBAC: Adding query filter params (role.id=a37e92d9517b45bf9dcfe085f74b17cf, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=b426ebd709e942819f19926f2e3668d2) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:12.968408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:12.969820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:12.975514 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 549/1100] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/role_assignments?user.id=b426ebd709e942819f19926f2e3668d2&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=a37e92d9517b45bf9dcfe085f74b17cf => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:13.052231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a24698-72e8-410e-b688-8e444adde63a None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:13.089859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:13.090559 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:13.091150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:13.091150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:13.102108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:13.107378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:13.108363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:13.109226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 552/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:34:13.115027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:13.218990 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7f521-4f2a-4b18-8164-36dc151a7798 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:13.253447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:13.254146 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:13.254431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:13.254715 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:13.255424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:13.255731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:13.257086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:13.258494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:13.267109 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 550/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:13.273162 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:13.291725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:13.376432 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f70e3-bcb7-4bcc-95b5-1dd44d792e12 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:13.410021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:13.410653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:13.410932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:13.411203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] PATH_INFO: `/v3/users/eef62dd6b34042c88ed02d179d7a65f9` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:13.412365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] RBAC: Authorizing `identity:get_user(user_id=eef62dd6b34042c88ed02d179d7a65f9)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:13.413791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:13.415426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 553/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:13 2022] GET /identity/v3/users/eef62dd6b34042c88ed02d179d7a65f9 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:13.421910 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:13.526923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeaebb-b7f6-407f-a426-44beac9ce020 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:13.562218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:13.562840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:13.563120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:13.563394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:13.624787 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:13.631678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:13.633118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:13.738797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 551/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:13.744747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:13.826938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:13.847035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c767-c6d7-448a-845a-272c41921c36 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:13.880394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:13.881036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:13.881320 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:13.881600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:13.882802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:13.884226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c2573-abf5-4ec9-bf56-f0001a7608af None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:13.885888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 554/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:13 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:13.892026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:13.981673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:13.987857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e92b-8f77-4241-979c-e9698f9f1de3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:14.019782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:14.020385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:14.020664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:14.020939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:14.021710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=94342ea7362f4a9b901f7f8071ae46b7, user.id=eef62dd6b34042c88ed02d179d7a65f9) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:14.022046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:14.023450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992d7b-4ebe-4211-b322-3fe4690cfbad None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:14.029641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 552/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:13 2022] GET /identity/v3/role_assignments?user.id=eef62dd6b34042c88ed02d179d7a65f9&scope.project.id=94342ea7362f4a9b901f7f8071ae46b7&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:16.017847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52bf70-8cb3-4d03-9b75-7e079a2dfda5 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:16.018401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52bf70-8cb3-4d03-9b75-7e079a2dfda5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:16.018401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52bf70-8cb3-4d03-9b75-7e079a2dfda5 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:16.019691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 555/1107] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:16.024855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:16.025752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:16.025994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:16.044133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:16.045327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:16.109686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 553/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:16.134239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52bf70-8cb3-4d03-9b75-7e079a2dfda5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:16.233634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52bf70-8cb3-4d03-9b75-7e079a2dfda5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:16.266238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:16.266866 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:16.267148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:16.267426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:16.268607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:16.269999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b52c6f-2f36-4f68-9768-f6bc094f5a40 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:16.271563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 556/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:16.277749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:16.364133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:16.374508 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfb11e-1065-44fb-b8e9-3e1ab0c46cfe None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:16.411518 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:16.412132 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:16.412415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:16.412695 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:16.413371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:16.414818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56671765-6ce6-4e2a-ae5f-382e021b9250 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:16.435361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 554/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:34:16 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:34:18.435359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f791-4f4a-4a95-a412-81a78504a616 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:18.435963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f791-4f4a-4a95-a412-81a78504a616 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:18.436078 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f791-4f4a-4a95-a412-81a78504a616 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:18.437393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 557/1111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:18.442608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:18.443027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:18.443167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:18.461208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:18.462424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:18.526284 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 555/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:18.548564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f791-4f4a-4a95-a412-81a78504a616 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:18.567413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:18.645783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f791-4f4a-4a95-a412-81a78504a616 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:18.678734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:18.679358 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:18.679634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:18.679906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] PATH_INFO: `/v3/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:18.681313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] RBAC: Authorizing `identity:get_role(role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:18.683292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:18.684614 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 558/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:18 2022] GET /identity/v3/roles/150d91154f9544c7a914d19f74d59fe8 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:18.690523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:18.764021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:18.787243 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287dd0-cd08-4de3-beef-ed7d3fe36604 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:18.819765 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:18.820399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:18.820685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:18.820965 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] PATH_INFO: `/v3/users/b0e38cc621004da5b01ac7e5be7b0064` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:18.834881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] RBAC: Authorizing `identity:get_user(user_id=b0e38cc621004da5b01ac7e5be7b0064)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:18.836298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:18.838582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 556/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:18 2022] GET /identity/v3/users/b0e38cc621004da5b01ac7e5be7b0064 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:18.844519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:18.926011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:18.941223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef939a-8714-436d-87a1-48ca00a52f7b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:18.973646 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:18.974277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:18.974563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:18.974844 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:19.039624 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:19.041083 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:19.121553 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:19.137356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 559/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:19.143107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:19.239198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc570dc-eef8-4123-91c5-94c0c8f4d810 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:19.272687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:19.273318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:19.273608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:19.273887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:19.275071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:19.276899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640d636-3af6-4e1f-9cfe-d34f96b57d56 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:19.278936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 557/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:19 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:19.284911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:19.382820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878136-1b56-4aa1-8fef-dc68b8c7ffad None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:19.414932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:19.415530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:19.416378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:19.416655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:19.417510 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=94342ea7362f4a9b901f7f8071ae46b7, user.id=b0e38cc621004da5b01ac7e5be7b0064) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:19.417818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:19.419260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4b4c6-2ed1-47fe-b296-5511cea78d29 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:19.424441 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 560/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:19 2022] GET /identity/v3/role_assignments?user.id=b0e38cc621004da5b01ac7e5be7b0064&scope.project.id=94342ea7362f4a9b901f7f8071ae46b7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:21.374554 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c43ee0-6965-404e-821e-d0343e2977a5 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:21.375121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c43ee0-6965-404e-821e-d0343e2977a5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:21.375121 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c43ee0-6965-404e-821e-d0343e2977a5 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:21.376450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 558/1118] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:21.381593 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:21.382055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:21.382170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:21.400425 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:21.401606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:21.445531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:21.466858 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 561/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:21.488884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c43ee0-6965-404e-821e-d0343e2977a5 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:21.491744 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:21.585313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c43ee0-6965-404e-821e-d0343e2977a5 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:21.617561 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:21.618202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:21.618480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:21.618782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] PATH_INFO: `/v3/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:21.620707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] RBAC: Authorizing `identity:get_role(role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:21.622081 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:21.623387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 559/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:21 2022] GET /identity/v3/roles/150d91154f9544c7a914d19f74d59fe8 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:21.629069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:21.723888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b65484-d5a6-4cdc-9c45-4f6bf16dffc8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:21.757287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:21.757917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:21.758227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:21.758504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] PATH_INFO: `/v3/users/b0e38cc621004da5b01ac7e5be7b0064` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:21.759872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] RBAC: Authorizing `identity:get_user(user_id=b0e38cc621004da5b01ac7e5be7b0064)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:21.761415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:21.763030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 562/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:21 2022] GET /identity/v3/users/b0e38cc621004da5b01ac7e5be7b0064 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:21.768716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:21.835652 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:21.865674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33abee-d187-4f73-b02a-d4219aa39fef None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:21.898389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:21.898967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:21.899252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:21.899530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:21.900920 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:21.902358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5dcb0-2730-4269-870b-f5f62c0a96cf None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:21.903997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 560/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:21 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:21.909737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:21.957338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:22.007465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb969890-de35-4d03-ab45-7077ef46e088 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:22.039894 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:22.040539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:22.040952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:22.041234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7/users/b0e38cc621004da5b01ac7e5be7b0064/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:22.043384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] RBAC: Authorizing `identity:create_grant(project_id=94342ea7362f4a9b901f7f8071ae46b7, user_id=b0e38cc621004da5b01ac7e5be7b0064, role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:22.044801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ddb01-6474-495c-81ed-8a2bdcef0256 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:22.058695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 563/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:34:21 2022] PUT /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7/users/b0e38cc621004da5b01ac7e5be7b0064/roles/150d91154f9544c7a914d19f74d59fe8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:34:24.006820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbab16-0042-4337-be89-28a8b5afdbe9 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.007325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbab16-0042-4337-be89-28a8b5afdbe9 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.007325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbab16-0042-4337-be89-28a8b5afdbe9 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.008627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 561/1124] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:24.014159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.014603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.014710 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.033371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:24.034583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:24.098108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 564/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:24.121117 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbab16-0042-4337-be89-28a8b5afdbe9 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:24.219798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbab16-0042-4337-be89-28a8b5afdbe9 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:24.252429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:24.253053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.253333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.253609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] PATH_INFO: `/v3/roles/150d91154f9544c7a914d19f74d59fe8` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.255044 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] RBAC: Authorizing `identity:get_role(role_id=150d91154f9544c7a914d19f74d59fe8)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:24.256394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:24.257758 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 562/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:24 2022] GET /identity/v3/roles/150d91154f9544c7a914d19f74d59fe8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:24.263984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:24.300857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:24.358366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2f53f-bc57-4815-84eb-a70341576309 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:24.391803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:24.392427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.392704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.392972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] PATH_INFO: `/v3/users/b0e38cc621004da5b01ac7e5be7b0064` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.394119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] RBAC: Authorizing `identity:get_user(user_id=b0e38cc621004da5b01ac7e5be7b0064)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:24.395523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:24.397108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 565/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:24 2022] GET /identity/v3/users/b0e38cc621004da5b01ac7e5be7b0064 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:24.402972 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:24.411592 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:24.500964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c12cd-28a9-4e61-a002-5b01d2da7b67 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:24.533528 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:24.534106 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.534390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.534669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.599251 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:24.600684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:24.609090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:24.696215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 563/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 06:34:24.702668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:24.798808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9d82-01ce-4459-b852-cbb72a9057e3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:24.805390 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:24.831917 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:24.832534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.832811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.833089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] PATH_INFO: `/v3/projects/94342ea7362f4a9b901f7f8071ae46b7` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.834238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] RBAC: Authorizing `identity:get_project(project_id=94342ea7362f4a9b901f7f8071ae46b7)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:24.835648 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642358e8-423c-4649-ad84-bd84d20a3cdb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:24.837275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 566/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:24 2022] GET /identity/v3/projects/94342ea7362f4a9b901f7f8071ae46b7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:24.843908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:24.939090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720972-072a-43a4-9834-05d2fa76fa99 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:24.946012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:24.972452 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:24.973050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:24.973329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:24.973600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:24.974394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] RBAC: Adding query filter params (role.id=150d91154f9544c7a914d19f74d59fe8, scope.project.id=94342ea7362f4a9b901f7f8071ae46b7, user.id=b0e38cc621004da5b01ac7e5be7b0064) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:24.974703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:24.976107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33784-d317-4a1c-b984-0b4a3d0b0add None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:24.981797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 564/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:24 2022] GET /identity/v3/role_assignments?user.id=b0e38cc621004da5b01ac7e5be7b0064&scope.project.id=94342ea7362f4a9b901f7f8071ae46b7&role.id=150d91154f9544c7a914d19f74d59fe8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:26.995319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28e3a-3d92-4851-b9d3-dc26a6d8ca6b None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:26.995834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28e3a-3d92-4851-b9d3-dc26a6d8ca6b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:26.995834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28e3a-3d92-4851-b9d3-dc26a6d8ca6b None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:26.997139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 567/1131] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:27.002221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:27.002623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:27.002770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:27.020887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:27.022101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:27.085774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 565/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:27.109372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb28e3a-3d92-4851-b9d3-dc26a6d8ca6b None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:27.206458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb28e3a-3d92-4851-b9d3-dc26a6d8ca6b None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:27.238514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:27.239128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:27.239408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:27.239683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:27.240804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:27.242232 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be428daa-06e7-41b8-abfa-e2e8155910ba None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:27.244291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 568/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:27.250017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:27.342094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:27.345255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457ce3a-4408-4ab8-a11b-83df38db9a4f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:27.378656 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:27.379267 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:27.379551 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:27.379832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:27.380505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:27.381923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ede156-bef5-4ddb-bd27-e0652dd07a5e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:27.408210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 566/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:34:27 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:34:29.390098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102005c-0014-40b7-bf9a-6a723445afd1 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:29.390633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102005c-0014-40b7-bf9a-6a723445afd1 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:29.390633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102005c-0014-40b7-bf9a-6a723445afd1 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:29.391977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 569/1135] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:29.397011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:29.397440 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:29.397571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:29.415730 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:29.417058 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:29.482244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 567/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:29.506834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102005c-0014-40b7-bf9a-6a723445afd1 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:29.549389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:29.606250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102005c-0014-40b7-bf9a-6a723445afd1 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:29.641668 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:29.642261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:29.642543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:29.642821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:29.643958 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:29.645376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e10c2-8764-4362-8fd7-df25746eb3b3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:29.646972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 570/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:29.652929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:29.731886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:29.751323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db19cb-de41-49b7-b3b2-2cb185411d26 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:29.784976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:29.785589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:29.785869 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:29.786175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:29.786859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:29.788276 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83f4fc-d2a3-4798-8b06-765efd75eff0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:29.809783 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 568/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:34:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:34:31.826900 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4c62-af95-4ade-b22f-d71640ab1b10 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:31.827426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4c62-af95-4ade-b22f-d71640ab1b10 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:31.827426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4c62-af95-4ade-b22f-d71640ab1b10 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:31.828793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 571/1139] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:31.833993 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:31.834387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:31.834547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:31.853916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:31.855130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:31.919258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 569/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:31.941086 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec4c62-af95-4ade-b22f-d71640ab1b10 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.038928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec4c62-af95-4ade-b22f-d71640ab1b10 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.050393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:32.074464 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.075090 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.075371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.075645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.085403 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:32.091331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:32.092501 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:32.093706 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 572/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:32.098565 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.162814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:32.195175 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3624238-bf98-4c26-912a-da5a4847634c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.228510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.229120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.229406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.229685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.230426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:32.230737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:32.232094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:32.240123 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 570/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:32.246332 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.344767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82274-83b1-443c-827c-92fb4f9525f3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.378227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.378849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.379131 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.379407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] PATH_INFO: `/v3/users/520d4cef5d734466a5f6967fc6fc436c` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.392226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] RBAC: Authorizing `identity:get_user(user_id=520d4cef5d734466a5f6967fc6fc436c)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:32.393661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:32.395319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 573/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:32 2022] GET /identity/v3/users/520d4cef5d734466a5f6967fc6fc436c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:32.401030 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.498275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2154a-d6db-4298-a93c-91e735338868 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.525578 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:32.531492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.532101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.532381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.532655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.596905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:32.598359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:32.694283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 571/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:32.700187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.728362 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:32.797836 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1cec-b793-4f06-8582-c9635eaeae8d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.830434 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.831054 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.831331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.831604 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] PATH_INFO: `/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.832727 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] RBAC: Authorizing `identity:get_project(project_id=16f6e1f521f94c6c9fa5a7ca5ece5d21)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:32.834163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec57141-9939-4da3-9a12-b60972b665e6 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:32.835787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 574/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:32 2022] GET /identity/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:32.842437 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:32.871066 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:32.938444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e136-bc8f-4f30-a3e7-758e6d04d689 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:32.971367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:32.971971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:32.972253 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:32.972534 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:32.973307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=16f6e1f521f94c6c9fa5a7ca5ece5d21, user.id=520d4cef5d734466a5f6967fc6fc436c) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:32.973619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:32.975069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2039822-33e3-4de5-a237-b4de285eca4c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:32.980899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 572/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:32 2022] GET /identity/v3/role_assignments?user.id=520d4cef5d734466a5f6967fc6fc436c&scope.project.id=16f6e1f521f94c6c9fa5a7ca5ece5d21&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:34.926673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4052-ccc4-4ff6-84b0-3213b57233c4 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:34.927195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4052-ccc4-4ff6-84b0-3213b57233c4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:34.927195 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4052-ccc4-4ff6-84b0-3213b57233c4 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:34.928504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 575/1147] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:34.934439 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:34.934882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:34.934977 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:34.953094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:34.954333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:35.018552 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 573/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:35.040752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4052-ccc4-4ff6-84b0-3213b57233c4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:35.133561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:35.140200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4052-ccc4-4ff6-84b0-3213b57233c4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:35.172477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:35.173094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:35.173370 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:35.173642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:35.183876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:35.188637 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:35.189729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:35.190628 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 576/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:35.195610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:35.218703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:35.288521 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d40dc-2570-4231-b3d4-29003f999e7a None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:35.322311 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:35.322923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:35.323203 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:35.323477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:35.324177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:35.324484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:35.325828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:35.333523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 574/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:35.339315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:35.436421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffa275-073b-4793-9fc8-e628c618cbf0 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:35.469378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:35.470025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:35.470312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:35.470591 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] PATH_INFO: `/v3/users/520d4cef5d734466a5f6967fc6fc436c` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:35.471731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] RBAC: Authorizing `identity:get_user(user_id=520d4cef5d734466a5f6967fc6fc436c)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:35.473150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:35.474749 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 577/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:35 2022] GET /identity/v3/users/520d4cef5d734466a5f6967fc6fc436c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:35.481145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:35.570515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:35.585193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539931-d0aa-488f-acbe-8118a93988e5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:35.618692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:35.619336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:35.619618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:35.620047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] PATH_INFO: `/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:35.621192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] RBAC: Authorizing `identity:get_project(project_id=16f6e1f521f94c6c9fa5a7ca5ece5d21)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:35.622637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dc628-84e0-4aae-ac8d-d1572d2772dc None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:35.624288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 575/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:35 2022] GET /identity/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:35.630310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:35.656443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:35.728458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd934b-cf2b-477d-af25-b5acef1fb175 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:35.759879 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:35.760523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:35.760802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:35.761074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] PATH_INFO: `/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21/users/520d4cef5d734466a5f6967fc6fc436c/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:35.762980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] RBAC: Authorizing `identity:create_grant(project_id=16f6e1f521f94c6c9fa5a7ca5ece5d21, user_id=520d4cef5d734466a5f6967fc6fc436c, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:35.764388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fc484-da2f-430d-8415-a2e7376b45bd None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:35.777081 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 578/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:34:35 2022] PUT /identity/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21/users/520d4cef5d734466a5f6967fc6fc436c/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:34:37.720134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ce01-0f97-43b7-ac6f-74cf142c812b None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:37.720627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ce01-0f97-43b7-ac6f-74cf142c812b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:37.720756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ce01-0f97-43b7-ac6f-74cf142c812b None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:37.722047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 576/1154] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:37.727018 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:37.727394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:37.727579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:37.746016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:37.747203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:37.811871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 579/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:37.834575 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ce01-0f97-43b7-ac6f-74cf142c812b None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:37.931690 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ce01-0f97-43b7-ac6f-74cf142c812b None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:37.963026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:37.963643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:37.963922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:37.964196 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:37.974145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:37.978962 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:37.980129 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:37.981202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 577/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:37.986227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:38.001461 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:38.083835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb1081-31c9-451e-8f4c-7cb13645b09d None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:38.117091 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:38.117694 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:38.118008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:38.118936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:38.119842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:38.120157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:38.121513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:38.129340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 580/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:38.135124 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:38.150724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:38.232494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4383a-2da8-4a05-9ad5-672a6caf20e3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:38.264738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:38.265364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:38.265647 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:38.265925 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] PATH_INFO: `/v3/users/520d4cef5d734466a5f6967fc6fc436c` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:38.267105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] RBAC: Authorizing `identity:get_user(user_id=520d4cef5d734466a5f6967fc6fc436c)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:38.268523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:38.270138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 578/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:38 2022] GET /identity/v3/users/520d4cef5d734466a5f6967fc6fc436c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:38.276033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:38.350309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:38.374457 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29458e-7bbc-45d0-8daf-853808153d83 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:38.407058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:38.407658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:38.407943 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:38.408215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:38.474307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:38.475737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:38.547810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:38.570204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 581/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:38.576228 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:38.656796 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:38.675241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fe7d7-3a98-406c-98bd-3f418b2c7aea None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:38.707565 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:38.708201 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:38.708486 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:38.708766 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] PATH_INFO: `/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:38.709908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] RBAC: Authorizing `identity:get_project(project_id=16f6e1f521f94c6c9fa5a7ca5ece5d21)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:38.711365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692719-d3e2-4cf4-a08a-9ed733ada80c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:38.713761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 579/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:38 2022] GET /identity/v3/projects/16f6e1f521f94c6c9fa5a7ca5ece5d21 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:38.719885 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:38.815250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57d301-42d9-4c1e-910c-20831d2efff9 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:38.848082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:38.848681 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:38.848962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:38.849239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:38.850034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=16f6e1f521f94c6c9fa5a7ca5ece5d21, user.id=520d4cef5d734466a5f6967fc6fc436c) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:38.850346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:38.851754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d9ac-cd03-4161-9ddd-2716c829447d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:38.858186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 582/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:38 2022] GET /identity/v3/role_assignments?user.id=520d4cef5d734466a5f6967fc6fc436c&scope.project.id=16f6e1f521f94c6c9fa5a7ca5ece5d21&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:40.922862 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99b8c-e326-4e6c-a8bf-624b957ea680 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:40.923365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99b8c-e326-4e6c-a8bf-624b957ea680 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:40.923365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f99b8c-e326-4e6c-a8bf-624b957ea680 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:40.924696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 580/1162] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:40.929780 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:40.930205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:40.930369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:40.948851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:40.950061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:40.983755 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:41.017758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 583/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:41.039978 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f99b8c-e326-4e6c-a8bf-624b957ea680 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:41.137823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f99b8c-e326-4e6c-a8bf-624b957ea680 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:41.169573 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:41.170208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:41.170487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:41.170755 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:41.188499 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:41.234806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:41.236488 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:41.328433 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 581/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:41.334454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:41.429747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ca858-fbe4-4b27-b5a0-c47a53b494c6 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:41.463714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:41.464280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:41.464562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:41.464842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:41.471293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:41.472715 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-66cfb568-6d0b-4f58-8b7e-81cfb7171a32 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:41.484711 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:41.486248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:41.487310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 584/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:34:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:41.492054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:41.538947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:41.587440 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcba3c-ebb7-474e-8431-fa837973df01 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:41.620483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:41.621093 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:41.621375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:41.621651 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:41.622373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:41.622681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:41.624061 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9797e0d-a09d-4447-a45e-f0bebba9805e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:41.631026 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 582/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jul 27 06:34:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:43.601733 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1acc7-a3ec-4a4f-b5f1-ca0ff1dd9cc4 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:43.602851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1acc7-a3ec-4a4f-b5f1-ca0ff1dd9cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:43.602851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1acc7-a3ec-4a4f-b5f1-ca0ff1dd9cc4 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:43.604348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 585/1167] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:43.609484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdecd1-b116-4bae-ab5d-403651bd9b7b None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:43.609883 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdecd1-b116-4bae-ab5d-403651bd9b7b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:43.610326 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdecd1-b116-4bae-ab5d-403651bd9b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:43.628331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-79cdecd1-b116-4bae-ab5d-403651bd9b7b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:43.629496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-79cdecd1-b116-4bae-ab5d-403651bd9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:43.696102 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 583/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:43.718897 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1acc7-a3ec-4a4f-b5f1-ca0ff1dd9cc4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:43.749397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:43.818423 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1acc7-a3ec-4a4f-b5f1-ca0ff1dd9cc4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:43.851194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:43.851842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:43.852071 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:43.852859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:43.853563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:43.854929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c55f20-002f-4276-b477-14622f046b31 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:43.870535 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 586/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Jul 27 06:34:43 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:34:45.867232 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af408fd-4901-4f19-b3e0-bc4580791814 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:45.867753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af408fd-4901-4f19-b3e0-bc4580791814 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:45.867753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af408fd-4901-4f19-b3e0-bc4580791814 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:45.869172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 584/1170] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:45.874409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:45.874790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:45.874972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:45.896920 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:45.898129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:45.963146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 587/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:45.986937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af408fd-4901-4f19-b3e0-bc4580791814 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:46.032353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:46.085171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af408fd-4901-4f19-b3e0-bc4580791814 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:46.117998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:46.118627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:46.118903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:46.119176 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] PATH_INFO: `/v3/domains/fd271be1bbb14afba3ec0b2db3247c3c` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:46.129083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fd271be1bbb14afba3ec0b2db3247c3c)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:46.130532 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833eae3-599f-49ba-80cc-7491b040f102 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:46.132112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 585/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 06:34:45 2022] GET /identity/v3/domains/fd271be1bbb14afba3ec0b2db3247c3c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:46.137839 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:46.158725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:46.237534 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772defd2-dc77-4830-ae74-fc5f959a9d56 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:46.269884 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:46.270510 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:46.270788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:46.271060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:46.271734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:46.273138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576524-afa7-4db5-9500-0de7804ae582 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:46.299424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 588/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:34:46 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 06:34:48.296526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37b36-5f6d-457e-a103-3cc54e5e6dd8 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:48.297157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37b36-5f6d-457e-a103-3cc54e5e6dd8 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:48.297157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37b36-5f6d-457e-a103-3cc54e5e6dd8 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:48.298461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 586/1174] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:48.303531 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:48.303979 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:48.304082 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:48.321609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:48.322811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:48.386794 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 589/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:34:48.411998 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b37b36-5f6d-457e-a103-3cc54e5e6dd8 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:48.510731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b37b36-5f6d-457e-a103-3cc54e5e6dd8 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:48.512936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:48.543876 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:48.544504 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:48.544786 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:48.545064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] PATH_INFO: `/v3/domains/fd271be1bbb14afba3ec0b2db3247c3c` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:48.546230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=fd271be1bbb14afba3ec0b2db3247c3c)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:48.547685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d3bb2-f03e-49de-88fc-c7cf8bd239fd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:48.549194 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 587/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jul 27 06:34:48 2022] GET /identity/v3/domains/fd271be1bbb14afba3ec0b2db3247c3c => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:48.554940 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:48.557472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:48.653416 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455f3f1-1d42-4ff9-b6aa-9ad638919d01 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:48.686334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:48.686951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:48.687233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:48.687511 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:48.688187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:48.689612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bf374-fcc7-4d9e-bd94-a2aa989afe43 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:48.710697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 590/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:34:48 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:34:50.718333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855edd-a892-4a8f-a36f-f1049b93b6cd None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:50.718863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855edd-a892-4a8f-a36f-f1049b93b6cd None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:50.718863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855edd-a892-4a8f-a36f-f1049b93b6cd None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:50.720227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 588/1178] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:50.725487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:50.725876 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:50.726097 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:50.744751 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:50.745932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:50.797427 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:50.811350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 591/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:50.834370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46855edd-a892-4a8f-a36f-f1049b93b6cd None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:50.932855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46855edd-a892-4a8f-a36f-f1049b93b6cd None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:50.965681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:50.966329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:50.966614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:50.966890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:50.976469 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:50.981339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:50.982549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:50.983835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 589/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:50.988562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:51.085704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aef492-c4da-479c-9d4f-beab36509fe5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:51.121107 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:51.121709 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:51.122011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:51.122293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:51.122994 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:51.123296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:51.124636 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:51.132341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 592/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:51.138674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:51.169225 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:51.235291 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352058b-b4c9-495c-a2cb-d3a52c8d2125 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:51.268134 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:51.268759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:51.269037 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:51.269309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] PATH_INFO: `/v3/users/310b674e800d441d871d92daebe73ece` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:51.281611 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] RBAC: Authorizing `identity:get_user(user_id=310b674e800d441d871d92daebe73ece)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:51.283059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:51.284632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 590/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:51 2022] GET /identity/v3/users/310b674e800d441d871d92daebe73ece => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:51.290285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:51.338975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:51.390592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac11f-48e8-4f74-a5ba-507b39116e25 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:51.425999 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:51.426617 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:51.426903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:51.427181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:51.492271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:51.494188 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:51.541065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:51.589952 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 593/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:51.595831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:51.685107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:51.695857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f044a1-8ef7-4640-9dbb-8eb819758556 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:51.729445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:51.730166 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:51.730453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:51.730729 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] PATH_INFO: `/v3/projects/74d223128bf34a069b2773c5f1541c24` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:51.731873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] RBAC: Authorizing `identity:get_project(project_id=74d223128bf34a069b2773c5f1541c24)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:51.733281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43cb2-8dfc-40b0-b30e-cf5579e260f1 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:51.735247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 591/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:51 2022] GET /identity/v3/projects/74d223128bf34a069b2773c5f1541c24 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:51.741281 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:51.837076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ea5fc-f74a-44e9-9aac-c3137f731f4d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:51.870487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:51.871073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:51.871350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:51.871620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:51.872393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=74d223128bf34a069b2773c5f1541c24, user.id=310b674e800d441d871d92daebe73ece) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:51.872698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:51.874138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c3e73-a154-4871-8c82-fe5eb7243b08 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:51.879293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 594/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:51 2022] GET /identity/v3/role_assignments?user.id=310b674e800d441d871d92daebe73ece&scope.project.id=74d223128bf34a069b2773c5f1541c24&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:53.820627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47c0d-3085-4c15-8f8d-55b409204398 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:53.821905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47c0d-3085-4c15-8f8d-55b409204398 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:53.821905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47c0d-3085-4c15-8f8d-55b409204398 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:53.823133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 592/1186] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:53.828282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:53.828717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:53.828846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:53.847629 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:53.848814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:53.858014 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:53.915619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 595/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:53.937684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47c0d-3085-4c15-8f8d-55b409204398 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:54.035011 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47c0d-3085-4c15-8f8d-55b409204398 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:54.067493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:54.068119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:54.068401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:54.068674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:54.078903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:54.083742 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:54.084838 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:34:54.085968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 593/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:54.091720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:54.189555 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e625a83-4f6c-4479-a83e-be2b6406d7f9 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:54.221613 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:54.222246 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:54.222525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:54.222798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:54.223500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:54.223804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:54.225140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:54.232826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 596/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:54.239001 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:54.259980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:54.337785 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae10700-bbf2-4305-9c15-e5951157de8b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:54.374847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:54.375479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:54.375756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:54.376030 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] PATH_INFO: `/v3/users/310b674e800d441d871d92daebe73ece` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:54.377164 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] RBAC: Authorizing `identity:get_user(user_id=310b674e800d441d871d92daebe73ece)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:54.378609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:54.380192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 594/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:54 2022] GET /identity/v3/users/310b674e800d441d871d92daebe73ece => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:54.387021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:54.396143 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:54.485310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80a4b-253a-4ed7-b92a-19ec1221c186 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:54.518470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:54.519096 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:54.519378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:54.519656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] PATH_INFO: `/v3/projects/74d223128bf34a069b2773c5f1541c24` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:54.520793 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] RBAC: Authorizing `identity:get_project(project_id=74d223128bf34a069b2773c5f1541c24)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:54.522237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca73c2b-f457-4370-af53-89b4f1e54af5 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:54.523916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 597/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:54 2022] GET /identity/v3/projects/74d223128bf34a069b2773c5f1541c24 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:54.529762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:54.611892 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:54.626713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c399e1e-f42e-4840-b6ae-f2e6d4027d9b None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:54.658864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:54.659533 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:54.659818 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:54.660096 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] PATH_INFO: `/v3/projects/74d223128bf34a069b2773c5f1541c24/users/310b674e800d441d871d92daebe73ece/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:54.662083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] RBAC: Authorizing `identity:create_grant(project_id=74d223128bf34a069b2773c5f1541c24, user_id=310b674e800d441d871d92daebe73ece, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:54.663508 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921cf3-24aa-42e3-8a7d-c19e36e89253 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:54.676718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 595/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:34:54 2022] PUT /identity/v3/projects/74d223128bf34a069b2773c5f1541c24/users/310b674e800d441d871d92daebe73ece/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:34:56.672391 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18abdf7-1a95-4ae7-aaa4-d0f9e50e1a83 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:56.672821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18abdf7-1a95-4ae7-aaa4-d0f9e50e1a83 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:56.672940 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18abdf7-1a95-4ae7-aaa4-d0f9e50e1a83 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:56.674341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 598/1193] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:34:56.679461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:56.679887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:56.680065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:56.698686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:34:56.699868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:34:56.765724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 596/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:34:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:34:56.788908 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18abdf7-1a95-4ae7-aaa4-d0f9e50e1a83 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:56.865304 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:56.891748 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18abdf7-1a95-4ae7-aaa4-d0f9e50e1a83 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:56.925923 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:56.926519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:56.926796 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:56.927068 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:56.937925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:34:56.942788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:34:56.943886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:34:56.945112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 599/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:34:56.950540 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:57.023966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:57.048629 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b801f8c-ee03-4dc5-ac53-a7b9f910e5b2 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:57.082514 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:57.083133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:57.083416 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:57.083696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:57.084403 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:57.084713 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:57.086110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:57.093859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 597/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:57.099722 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:57.204522 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc3b0-863b-4a14-befc-c34a8dfee490 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:57.238295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:57.238925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:57.239205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:57.239480 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] PATH_INFO: `/v3/users/310b674e800d441d871d92daebe73ece` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:57.240623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] RBAC: Authorizing `identity:get_user(user_id=310b674e800d441d871d92daebe73ece)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:57.242067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:57.243692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 600/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jul 27 06:34:57 2022] GET /identity/v3/users/310b674e800d441d871d92daebe73ece => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:57.249567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:57.344140 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb32c1-a738-4744-813c-2b6196bb3872 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:57.376145 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:57.376746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:57.377023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:57.377295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:57.385387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:57.441804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:57.443252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:57.536790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 598/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:34:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:34:57.542864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:57.545684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:57.642006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd95cde-3747-4f63-acca-2f1ceeba1e7d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:57.675268 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:57.675910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:57.676194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:57.676472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] PATH_INFO: `/v3/projects/74d223128bf34a069b2773c5f1541c24` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:57.677623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] RBAC: Authorizing `identity:get_project(project_id=74d223128bf34a069b2773c5f1541c24)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:57.679093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329128-dd96-4e7c-acaa-daa6a956f821 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:57.681532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 601/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jul 27 06:34:57 2022] GET /identity/v3/projects/74d223128bf34a069b2773c5f1541c24 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:34:57.687596 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:34:57.725219 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:34:57.799353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3398-b6c7-4a68-a7bc-257f35142ff4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:34:57.831992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:34:57.832597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:34:57.832880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:34:57.833157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:34:57.833934 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=74d223128bf34a069b2773c5f1541c24, user.id=310b674e800d441d871d92daebe73ece) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:34:57.834272 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:34:57.835688 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1e6eb-d5a9-4d69-8eec-dd080c443a7f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:34:57.841424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 599/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:34:57 2022] GET /identity/v3/role_assignments?user.id=310b674e800d441d871d92daebe73ece&scope.project.id=74d223128bf34a069b2773c5f1541c24&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:07.671128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=65 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:07.838860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=49 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:52.634670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7da4-55b2-4349-9009-de8d89b943e0 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:52.635284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7da4-55b2-4349-9009-de8d89b943e0 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:52.635284 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7da4-55b2-4349-9009-de8d89b943e0 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:52.636583 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 602/1201] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:35:52.643005 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:52.643465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:52.643577 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:52.669762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:35:52.671597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:35:52.760111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 600/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:35:52.786842 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb7da4-55b2-4349-9009-de8d89b943e0 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:52.911296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb7da4-55b2-4349-9009-de8d89b943e0 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:52.953272 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:52.953907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:52.954252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:52.954555 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:52.967459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:52.968961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:35:52.984954 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:52.989631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:52.990707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:35:52.990707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:35:52.990707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:52.990707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:35:52.990707 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 603/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:35:52.992950 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:53.125591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ace1d-1083-4ed8-bb93-4d309fcc0e21 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:53.167325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:53.174477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:53.179204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:53.179490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:53.179770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:53.180509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:53.180801 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:53.182213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332f65-7a44-4c61-aa50-c380c6afd8ea None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:53.196142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 601/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:53.203165 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:53.273072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:53.325890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df9e99-5727-40b3-8723-7953cb1fc38e None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:53.387468 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:53.388139 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:53.388421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:53.388699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:53.389518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:53.391060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aaf97-f113-41cf-9d61-cb337d697e5c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:53.418448 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 604/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jul 27 06:35:53 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:35:55.751165 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605511-0961-4f67-8823-477fa3990ff3 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:55.751730 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605511-0961-4f67-8823-477fa3990ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:55.751730 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605511-0961-4f67-8823-477fa3990ff3 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:55.753252 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 602/1206] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:35:55.759248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:55.759765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:55.759864 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:55.779654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:35:55.781127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:35:55.862450 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 605/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:35:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:35:55.886407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605511-0961-4f67-8823-477fa3990ff3 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.000734 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605511-0961-4f67-8823-477fa3990ff3 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.039992 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.040633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.040907 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.041226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.057931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:35:56.063576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:56.064574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:35:56.065490 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 603/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:35:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:35:56.071327 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.116487 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:56.184782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2245b4-057a-4955-9187-7998bf5c571f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.227401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.228050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.228307 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.228581 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.229297 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:56.229606 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:56.231006 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:56.239872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 606/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:56.246545 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.303043 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:56.364396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdf3ac-f10b-4e71-a0b9-d21719615963 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.405774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.407070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.407070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.407070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.420682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:56.422396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:35:56.442626 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:56.444429 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:56.445627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:35:56.445627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:35:56.445627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:56.445627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:35:56.445627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 604/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:35:56.450496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.563996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:56.567127 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86de26-5e1a-4c2c-a9a3-931b23d216c7 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.610688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.611414 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.611763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.612140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.612909 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:56.613317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:56.614787 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:56.629716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 607/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:56.636369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.758510 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a934cb-fb71-42b4-b1b3-47c5767c70b2 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.760326 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:56.799325 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.800023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.800307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.800609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.809338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:56.810872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:35:56.822736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:56.823795 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:35:56.825363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 605/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:35:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:35:56.831051 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:56.947880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183932d-21a0-4d0b-9ff2-b2891afadeaf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:56.984783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:56.985653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:56.985933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:56.986241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:56.987035 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:56.987340 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:56.988756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:57.007791 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 608/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:35:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:57.013583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:57.133041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b91b-997c-46b5-8b33-776cc004bbc5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:57.171886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:57.172519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:57.172797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:57.173208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:57.204728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:57.248888 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:57.251023 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:57.359677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 606/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:35:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 06:35:57.367049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:57.388118 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:57.480951 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e509b6-ab1d-49d9-bdc7-9dd6aabbcd62 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:57.524223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:57.524986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:57.525345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:57.525696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:57.536635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:57.538684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:35:57.554817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:57.555997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:57.557022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:35:57.557022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:35:57.557022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:35:57.557022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:35:57.557022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 609/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:35:57.562218 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:57.626828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:57.679114 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a3b20-5726-4a06-9c8c-4c56f0cc58a7 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:57.719769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:57.720396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:57.720674 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:57.720953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:57.721660 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:57.721973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:57.723332 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:57.736837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 607/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:35:57.742520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:57.817222 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:57.853205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8890-963f-4aee-9d33-37c47464174d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:57.892064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:57.892722 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:57.893052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:57.893335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:57.901182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:57.902716 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:35:57.915899 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:35:57.916861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:35:57.917850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 610/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:35:57.923167 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:58.044104 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9be689-5ca1-423a-bc9b-54e503c5a2af None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:58.067193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:58.081610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:58.082358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:58.082709 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:58.083057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:58.083879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:58.084804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:58.086828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d604-004c-4fd8-a9c2-204e038f1739 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:58.100462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 608/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:35:58.107100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:35:58.217699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293a6b-1eaa-4eb9-b908-58d6af6d539b None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:35:58.252975 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:35:58.260325 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:35:58.260948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:35:58.261231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:35:58.261507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:35:58.262360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=5da5ce7121544c5f9838982de97b645c) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:35:58.263219 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:35:58.264213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be28831-b7a7-429b-bc6e-c9513f6febb0 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:35:58.270684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 611/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:35:58 2022] GET /identity/v3/role_assignments?user.id=5da5ce7121544c5f9838982de97b645c&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:36:00.297774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec02480-f850-4f91-9122-2a1805a4433c None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:00.298358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec02480-f850-4f91-9122-2a1805a4433c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:00.298453 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec02480-f850-4f91-9122-2a1805a4433c None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:00.299841 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 609/1220] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:00.305772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:00.306295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:00.306397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:00.325580 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:00.326798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:00.399865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 612/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:00.427208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec02480-f850-4f91-9122-2a1805a4433c None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:00.549339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec02480-f850-4f91-9122-2a1805a4433c None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:00.590544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:00.591328 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:00.591646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:00.591944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:00.609981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:00.611604 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:00.612856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 610/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:00.618750 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:00.727072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429d31c-11e1-45d3-a461-19611af766d6 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:00.751929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:00.763654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:00.764331 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:00.764621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:00.764902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:00.765623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] RBAC: Adding query filter params (name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:00.765934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:00.767344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:00.776682 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 613/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:00.784602 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:00.836882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:00.895059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8367821-ff31-4125-9bd5-c4c182562290 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:00.931259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:00.931914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:00.932190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:00.932466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:00.942370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:00.943854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:00.958244 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:00.960012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:00.961033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:00.961033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:00.961033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:00.961033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:00.961033 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 611/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:00.968028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.075934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb8c2-b5ca-4e2f-8d71-bd6d9c3e41cb None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.109608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:01.110223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:01.110843 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:01.111080 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:01.111841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:01.112151 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:01.113497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:01.126201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 614/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:01.132492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.235609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:01.243112 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a5542-0ddf-4f91-b049-24706008d5f9 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.276979 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:01.277607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:01.277886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:01.278190 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:01.285697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:01.287250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:36:01.298363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:01.299971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:01.301136 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 612/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:01.306634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.361708 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:01.437688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83044-8949-46d3-b3ee-84f1cac9c7ff None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.475735 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:01.476355 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:01.476672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:01.476888 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:01.477664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:01.477994 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:01.479516 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:01.498345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 615/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:36:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:01.504323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.605117 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db964-e339-483d-91f0-8b9686880c74 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.640798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:01.641367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:01.641646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:01.641919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:01.654465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:01.655964 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:01.675657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:01.676776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:01.678448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:01.678448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:01.678448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:01.678448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:01.678448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 613/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:01.683687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.791695 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5a16e-27a5-4d93-a25e-88aa05d5051d None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.794200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:01.828674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:01.829294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:01.829620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:01.829905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:01.830652 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:01.831015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:01.832430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:01.845612 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 616/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:01.851853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:01.861770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:01.963628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7113ab1-32db-4dbc-891b-b3bbe21a2379 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:01.999497 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:02.000230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:02.000541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:02.000899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:02.007924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:02.009421 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:36:02.020812 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:02.022537 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:02.023648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 614/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:02.028623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:02.137994 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e2ea9-d564-4a9c-9070-012346428adf None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:02.176659 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:02.177445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:02.177736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:02.178614 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:02.179635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:02.180036 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:02.181846 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a1d6-c65f-41dc-8de2-c5700393859b None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:02.195644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 617/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:02.201960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:02.278980 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:02.316017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394d01-18dc-4948-99d4-8b09685dce99 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:02.351767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:02.352422 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] REQUEST_METHOD: `PUT` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:02.352702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:02.352960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/5da5ce7121544c5f9838982de97b645c/roles/230bd8e022eb4b6f88782d3f9c516f52` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:02.369737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=5da5ce7121544c5f9838982de97b645c, role_id=230bd8e022eb4b6f88782d3f9c516f52)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:02.371239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c436-7067-4951-b33c-293a97b0f307 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:02.385213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 615/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:36:02 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/5da5ce7121544c5f9838982de97b645c/roles/230bd8e022eb4b6f88782d3f9c516f52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 06:36:04.468033 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd93eec-e9d5-4bfd-a373-508bba15d2b7 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:04.468752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd93eec-e9d5-4bfd-a373-508bba15d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:04.468752 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd93eec-e9d5-4bfd-a373-508bba15d2b7 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:04.469969 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 618/1233] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:04.476020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:04.476470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:04.476564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:04.496581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:04.498202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:04.568015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 616/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:36:04.593281 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd93eec-e9d5-4bfd-a373-508bba15d2b7 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:04.623541 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:04.711938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd93eec-e9d5-4bfd-a373-508bba15d2b7 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:04.750335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:04.750881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:04.751163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:04.751438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:04.761879 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:36:04.766883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:04.768013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:04.769062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 619/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jul 27 06:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 06:36:04.774789 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:04.823318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:04.882028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1b54b-ad23-490b-9beb-aff8f6f66504 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:04.916705 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:04.917333 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:04.917612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:04.917889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:04.918639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] RBAC: Adding query filter params (name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:04.918942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:04.920283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:04.928944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 617/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jul 27 06:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:04.934587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:05.031928 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:05.048513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4e0-78d5-4d07-8d63-6f7d8835bc6c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:05.086829 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:05.087472 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:05.087753 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:05.088030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:05.098104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:05.100116 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:05.117674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:05.118910 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:05.119808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:05.119808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:05.119808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:05.119808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:05.119808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 620/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:05.125815 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:05.234428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb8ea5-1727-4854-b926-95f51e09c14c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:05.244859 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:05.275639 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:05.276267 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:05.276687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:05.276832 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:05.277527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:05.277850 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:05.279268 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:05.292893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 618/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:05.298546 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:05.412802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d70aef-1ac6-4f02-a780-e30119d525cc None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:05.453719 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:05.454913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:05.454913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:05.454913 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:05.457619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:05.463838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:05.465792 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:36:05.478563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:05.481062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:05.482285 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:05.484777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 621/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:05.486080 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:05.589455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650db57-3dcc-4201-98a6-505423fdbf38 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:05.625682 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:05.626338 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:05.626618 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:05.626890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:05.627677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:05.627984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:05.629387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:05.647616 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 619/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:36:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:05.653765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:05.773013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888262c-9f61-44ba-ac82-2a118450cca1 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:05.810371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:05.810997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:05.811281 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:05.811562 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:05.878383 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:05.886697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:05.888514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.002925 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 622/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:36:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:36:06.009353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:06.037733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:06.113718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820c8cc-0079-43ea-a2ef-2ba86c4e8554 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:06.150541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:06.151110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:06.151391 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:06.151664 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:06.163710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:06.165418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:06.185007 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:06.186304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:06.187209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:06.187209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:06.187209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:06.187209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:06.187209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 620/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:06.192860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:06.313764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:06.320074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6511a1-1d7c-4307-bbae-705a0854ec28 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:06.356579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:06.357201 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:06.357484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:06.357760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:06.358486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:06.358834 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:06.360207 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.371903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 623/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:06.377916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:06.473321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:06.495801 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621a216-f87b-4f87-ae15-38c0315dcf86 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:06.532004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:06.532724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:06.533680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:06.533987 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:06.541494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:06.543060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:36:06.556614 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:06.557563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:06.558723 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 621/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:06.564505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:06.680654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74782a78-2c68-4b59-8862-c8166a10184a None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:06.719013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:06.719682 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:06.719962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:06.720243 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:06.720991 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:06.721302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:06.722758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e253-7ddb-4800-b3f2-2e6f62ef28de None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.736236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 624/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:36:06.743669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:06.855526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700b607-7e6a-442b-807c-f246240d0b54 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:06.901445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:06.902113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:06.902395 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:06.902672 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:06.903456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] RBAC: Adding query filter params (role.id=230bd8e022eb4b6f88782d3f9c516f52, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=5da5ce7121544c5f9838982de97b645c) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:06.903777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:06.905198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f4096-893e-4177-bbfc-91e6079ae0a4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:06.911811 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 622/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:36:06 2022] GET /identity/v3/role_assignments?user.id=5da5ce7121544c5f9838982de97b645c&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=230bd8e022eb4b6f88782d3f9c516f52 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:36:09.157687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0f1-75f2-4a5b-9f83-400fdb9d0c5f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:09.158424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0f1-75f2-4a5b-9f83-400fdb9d0c5f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:09.158424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0f1-75f2-4a5b-9f83-400fdb9d0c5f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:09.159977 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 625/1247] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:09.165543 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:09.166174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:09.166174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:09.173221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:09.188355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:09.189971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:09.265174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 623/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:09.290124 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb0f1-75f2-4a5b-9f83-400fdb9d0c5f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:09.311818 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:09.413645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb0f1-75f2-4a5b-9f83-400fdb9d0c5f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:09.458592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:09.459533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:09.459533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:09.459820 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:09.471337 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:36:09.477916 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:09.479194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:09.480526 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:09.484366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 626/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:09.485828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:09.605598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d017e8-9d54-4190-8135-8b75b1904f55 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:09.637015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:09.642760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:09.643402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:09.643681 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:09.643961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:09.644685 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:09.644988 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:09.646517 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:09.655725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 624/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:09.662783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:09.750377 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:09.778417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9a5c6-b7a2-42bc-82ba-12b2b6a05c46 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:09.820800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:09.821538 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:09.821817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:09.822123 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:09.833874 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:09.835470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:09.852554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:09.853672 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:09.854860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:09.854860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:09.854860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:09.854860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:09.854860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 627/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:09.860208 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:09.988704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96404b-1c58-40c1-ba5f-59c0d2fa73a6 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:10.041366 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:10.042036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:10.042250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:10.042529 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:10.043238 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:10.043536 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:10.044966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:10.059728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 625/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:10.065604 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:10.192802 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef17c5-03ca-46de-85d9-25ddabe71440 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:10.226513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:10.236250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:10.237216 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:10.237492 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:10.237766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:10.251237 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:10.253047 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:36:10.266825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:10.268141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:10.269247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 628/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:10.275053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:10.337700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:10.395797 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760ffae-db76-4e5f-b172-e65767ae9588 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:10.441600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:10.442261 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:10.442564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:10.442823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:10.443689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:10.444092 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:10.445855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:10.464564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 626/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:36:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:10.470445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:10.595455 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260800-7da3-4c40-993d-f719e4cc2d65 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:10.641251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:10.641903 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:10.642223 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:10.642503 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:10.727041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:10.742039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:10.743641 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:10.861197 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 629/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:36:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 06:36:10.869515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:10.992576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452958cf-0384-4bab-8f57-52206cfc56bf None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:11.009549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:11.033389 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:11.034070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:11.034354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:11.034631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:11.048579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:11.050053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:11.072612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:11.073889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:11.075079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:11.075079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:11.075079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:11.075079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:11.075079 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 627/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:11.080206 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:11.192685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:11.209765 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a2eb5-41c7-42d7-b0e0-2ff79e8f0893 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:11.256407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:11.257197 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:11.257499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:11.257776 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:11.259348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:11.259348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:11.260364 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:11.276277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 630/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:11.282022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:11.407025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1939c7-d5fe-43f1-bf56-a3eb2d17699f None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:11.451223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:11.451805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:11.452090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:11.452369 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:11.459728 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:11.461363 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:36:11.473126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:11.474398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:11.475493 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 628/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:11.480517 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:11.591388 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c8d7-9946-4bb2-af24-eb3b92aacd65 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:11.630067 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:11.631481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:11.631775 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:11.632163 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:11.633037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:11.633438 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:11.635008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff420fe-a111-4894-a7e4-4b7138263c25 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:11.647843 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 631/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:11.654675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:11.688241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:11.770703 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d04167-c16c-4975-be41-ca79333912bb None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:11.806329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:11.807035 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:11.807326 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:11.807601 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:11.808377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=5da5ce7121544c5f9838982de97b645c) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:11.808767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:11.810309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3c84d-652c-496a-bd15-216e098bba7b None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:11.815779 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 629/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:36:11 2022] GET /identity/v3/role_assignments?user.id=5da5ce7121544c5f9838982de97b645c&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:13.852170 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37eaa-105a-4c5a-9af8-76652ca12543 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:13.852691 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37eaa-105a-4c5a-9af8-76652ca12543 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:13.852810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37eaa-105a-4c5a-9af8-76652ca12543 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:13.854098 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 632/1261] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:13.859316 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:13.859716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:13.859870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:13.878394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:13.879595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:13.943668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 630/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:13.965215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c37eaa-105a-4c5a-9af8-76652ca12543 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:13.976050 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.064639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c37eaa-105a-4c5a-9af8-76652ca12543 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.097963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.098602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.098887 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.099166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.109463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:36:14.114630 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:14.116488 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:14.117704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 633/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:14.122224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:14.139878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.218118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dbf6e-fee6-4664-9226-52aebc5f1206 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.251443 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.252059 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.252342 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.252619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.253329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:14.253636 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:14.255018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:14.262985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 631/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:14.268533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:14.333367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.366686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355d19d-32b2-44e3-a264-13f915a044be None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.407782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.410886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.410886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.410886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.424130 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:14.425790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:14.442449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:14.443664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:14.444773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:14.444773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:14.444773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:14.444773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:14.444773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 634/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:14.449898 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:14.528355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.547012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c435-6039-46b7-b690-eb8db5483199 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.578717 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.579318 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.579598 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.579870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.580544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:14.580847 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:14.582223 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:14.594236 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 632/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:14.599972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:14.697338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a89-918e-477a-9963-b4ae50f29c35 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.716561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.731099 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.731729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.732011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.732287 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.739660 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:14.741273 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:36:14.752184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:14.753267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:14.754539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 635/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:14.759322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:14.853822 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bbde9-176b-4128-995e-a9b21278a855 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:14.886401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:14.887008 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:14.887281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:14.887555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:14.888337 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:14.888642 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:14.890064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:14.898863 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:14.908126 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 633/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:14.913742 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:15.009280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3de16-4588-4054-9e8b-0c9299b9c382 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:15.041215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:15.041817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:15.042128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:15.042398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:15.052371 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:15.053996 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:15.067571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:15.069158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:15.070421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:15.070421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:15.070421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:15.070421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:15.070421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 636/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:15.074799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:15.170386 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841eddf-78a8-4b52-a9e1-8918f71e9ba3 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:15.202740 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:15.203336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:15.203613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:15.203886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:15.204555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:15.204856 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:15.206213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:15.216875 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 634/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:15.222402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:15.253959 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:15.323470 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1434efa-14d0-4d0a-ad1a-1b3092a552af None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:15.355805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:15.356430 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:15.356714 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:15.356989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:15.364040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:15.366392 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:36:15.378286 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:15.379319 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:15.380315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 637/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:15.385678 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:15.452407 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:15.494020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe236-72a4-4caa-9899-2bbdb5138cdd None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:15.526069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:15.526675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:15.526954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:15.527226 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:15.527952 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:15.528255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:15.529644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2eaed-f3dd-4c67-865e-a4c8b1c51415 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:15.541864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 635/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:15.547904 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:15.641406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:15.651228 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcb8d2-248f-4056-9513-f871d316a40c None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:15.684236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:15.684983 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:15.685290 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:15.685563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] PATH_INFO: `/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/5da5ce7121544c5f9838982de97b645c/roles/ec39c739616b4f71bcc574e09e75fbfa` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:15.687666 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] RBAC: Authorizing `identity:create_grant(project_id=85f26e9454984c23b5d9935e5a1e7109, user_id=5da5ce7121544c5f9838982de97b645c, role_id=ec39c739616b4f71bcc574e09e75fbfa)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:15.689077 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b1bf1-fa41-44c5-97bc-928334146aae None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:15.701871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 638/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jul 27 06:36:15 2022] PUT /identity/v3/projects/85f26e9454984c23b5d9935e5a1e7109/users/5da5ce7121544c5f9838982de97b645c/roles/ec39c739616b4f71bcc574e09e75fbfa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:36:17.732659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25281-67de-43f1-9cb6-4c628137e0df None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:17.733200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25281-67de-43f1-9cb6-4c628137e0df None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:17.733200 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25281-67de-43f1-9cb6-4c628137e0df None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:17.734607 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 636/1274] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:17.739830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:17.740747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:17.740747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:17.758277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:17.759479 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:17.824746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 639/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:17.848428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf25281-67de-43f1-9cb6-4c628137e0df None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:17.948455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf25281-67de-43f1-9cb6-4c628137e0df None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:17.966323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:17.983073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:17.983701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:17.983985 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:17.984264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:17.994472 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 06:36:17.999310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:18.000477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:18.001563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 637/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jul 27 06:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:18.007014 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.086644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:18.104558 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b6355-295a-4fe0-8bc5-b8afead3cea0 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.136838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.137442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.137719 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.138180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.138886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:18.139187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.140530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:18.148927 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 640/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:36:18.154558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.252090 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f473-1ba9-41f0-9282-bf83af8c540a None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.284899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.285520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.285799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.286105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.296852 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.298309 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:18.312317 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:18.313365 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:18.314576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:18.314576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:18.314576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:18.314576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:18.314576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 638/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:18.319519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.424738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03eb03-848f-485c-bd7a-95faf76ccf2f None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.461564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.462191 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.462477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.462756 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] PATH_INFO: `/v3/domains` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.463445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:18.463754 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.465140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:18.477219 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 641/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:18.482854 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.514101 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:18.580572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d4bd-b715-4942-a0d1-a9a6159f73d0 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.613320 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.613961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.614268 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.614548 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.622445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.623873 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 06:36:18.634752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:18.635849 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:18.636855 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 639/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:18.641557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.645263 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:18.741341 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e465307-e7a4-4790-a36e-01841a03c060 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.774426 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.774973 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.775257 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.775536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] PATH_INFO: `/v3/users` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.776329 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:18.776639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.778104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:18.794961 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 642/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:18.800662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:18.888213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:18.897961 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5a2d0-b2f6-4868-83b7-a001905dc1d4 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:18.930581 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:18.931249 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:18.931530 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:18.931805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:18.995872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:18.997294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.080154 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:19.090893 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 640/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jul 27 06:36:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 06:36:19.096661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:19.157491 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:19.194220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2954d-d59c-428b-b3e6-8f84750a5b50 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:19.227313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:19.227943 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:19.228226 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:19.228505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:19.238421 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:19.239911 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 06:36:19.254213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:19.255280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:19.260789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 06:36:19.260789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 06:36:19.260789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 06:36:19.260789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:19.260789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 643/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:19.261151 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:19.356770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5294-5953-4e10-aa3d-c8d304bdf27e None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:19.390589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:19.391271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:19.391560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:19.391864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:19.392642 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:19.393076 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:19.394889 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.410286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 641/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jul 27 06:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:19.416523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:19.513670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f45bd-81b6-4bc9-ad84-ec06170612d3 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:19.527231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:19.546620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:19.547238 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:19.547515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:19.547789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:19.554291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:19.555702 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 06:36:19.566665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:19.567744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: ERROR keystone.server.flask.application Jul 27 06:36:19.569732 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 644/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jul 27 06:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:19.574040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:19.621900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:19.670550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea651-ca0f-42c9-8530-8260c6d98fac None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:19.703687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba527-6f5a-472e-a668-21212481212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:19.704305 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba527-6f5a-472e-a668-21212481212d None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:19.704588 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba527-6f5a-472e-a668-21212481212d None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:19.704864 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba527-6f5a-472e-a668-21212481212d None admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:19.705608 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba527-6f5a-472e-a668-21212481212d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:19.705919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba527-6f5a-472e-a668-21212481212d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:19.707361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba527-6f5a-472e-a668-21212481212d None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.719013 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 642/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jul 27 06:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:19.725456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:19.822955 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cd42a-f99b-4d46-b237-f6b7beba6e23 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:19.855579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:19.856181 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:19.856462 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:19.856738 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:19.857507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] RBAC: Adding query filter params (role.id=ec39c739616b4f71bcc574e09e75fbfa, scope.project.id=85f26e9454984c23b5d9935e5a1e7109, user.id=5da5ce7121544c5f9838982de97b645c) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:19.857814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:19.859247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19c7c7-0509-4402-8681-12fff4a29f5f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:19.865529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 645/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jul 27 06:36:19 2022] GET /identity/v3/role_assignments?user.id=5da5ce7121544c5f9838982de97b645c&scope.project.id=85f26e9454984c23b5d9935e5a1e7109&role.id=ec39c739616b4f71bcc574e09e75fbfa => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:21.872027 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd53343-932f-4710-9728-00b3c023cff4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:21.872613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd53343-932f-4710-9728-00b3c023cff4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:21.872613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd53343-932f-4710-9728-00b3c023cff4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:21.873911 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 643/1288] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:21.878985 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:21.879446 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:21.879548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:21.898449 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:21.899618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:21.928849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:21.965773 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 646/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:21.987791 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd53343-932f-4710-9728-00b3c023cff4 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:22.089057 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd53343-932f-4710-9728-00b3c023cff4 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:22.099259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:22.122619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:22.123247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:22.123527 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:22.123801 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] PATH_INFO: `/v3/services/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:22.124423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:22.125769 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:22.134963 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:22.136217 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:22.137302 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 644/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:36:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:22.142141 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:22.237109 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66ffe4-8cdc-4fff-be4d-9316bc057bc1 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:22.270182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:22.270782 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:22.271094 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:22.271369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:22.272040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:22.272345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:22.273687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4bb5f-d17d-46f9-90b3-eeb37a9a1e14 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:22.278345 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 647/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 06:36:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:22.283942 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:22.380275 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9ba6-1d82-4955-9ebc-1fb407c7337c None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:22.420560 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:22.421430 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:22.421916 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:22.422442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] PATH_INFO: `/v3/services` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:22.423352 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:22.423711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:22.425587 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e7017-910a-4a8b-a683-d4c7d0b69b19 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:22.433410 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 645/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 06:36:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:24.406173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8d4c0-267c-4043-94bf-71a6d2af2cf9 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:24.406746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8d4c0-267c-4043-94bf-71a6d2af2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:24.406746 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8d4c0-267c-4043-94bf-71a6d2af2cf9 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:24.408071 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 648/1293] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:24.413807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c56-51f2-4d18-89ab-15fbd603c17c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:24.414294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c56-51f2-4d18-89ab-15fbd603c17c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:24.414397 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6c56-51f2-4d18-89ab-15fbd603c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:24.432267 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-b97b6c56-51f2-4d18-89ab-15fbd603c17c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:24.433462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-b97b6c56-51f2-4d18-89ab-15fbd603c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:24.462505 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:24.499686 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 646/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 06:36:24.522709 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8d4c0-267c-4043-94bf-71a6d2af2cf9 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:24.554217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:24.622861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8d4c0-267c-4043-94bf-71a6d2af2cf9 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:24.656148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:24.656807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:24.657038 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:24.657311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:24.658032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:24.659394 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19375ee3-c7af-4049-8549-9a91e3a04908 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:24.673785 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 649/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jul 27 06:36:24 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:36:26.667871 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2779-be3e-4ba5-b74e-78114d0a6194 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:26.668264 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2779-be3e-4ba5-b74e-78114d0a6194 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:26.668424 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2779-be3e-4ba5-b74e-78114d0a6194 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:26.669790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 647/1296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:26.675021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:26.675466 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:26.675576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:26.693080 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:26.694282 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:26.763634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 650/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:26.786233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b2779-be3e-4ba5-b74e-78114d0a6194 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:26.883174 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b2779-be3e-4ba5-b74e-78114d0a6194 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:26.915813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:26.916370 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:26.916646 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:26.916919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:26.917542 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:26.918922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:26.927595 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:26.928908 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:26.930078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 648/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:36:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:26.934608 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:26.952697 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:27.031253 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f184477-b1a2-46ff-8733-1da56a46dd50 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:27.062698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:27.063300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:27.063579 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:27.063849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:27.064514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:27.064815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:27.066180 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:27.071661 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 651/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 06:36:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 06:36:27.077585 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:27.108735 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:27.174905 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016106ba-1c78-42cf-9394-322f5dac25ad None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:27.207313 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:27.207924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:27.208209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:27.208550 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:27.209233 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] RBAC: Adding query filter params (interface=public, service_id=1afe1be0bc9d40d5ac9f11f4de18f61c, region_id=RegionOne) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:27.209545 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:27.210939 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c079e2-24fe-47de-8b37-66af96eabaf6 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:27.215759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 649/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jul 27 06:36:27 2022] GET /identity/v3/endpoints?service_id=1afe1be0bc9d40d5ac9f11f4de18f61c&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:27.221275 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:27.299153 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:27.317621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05975-94c1-4f65-aa98-0f2cf00fd163 None admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:27.350058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:27.350665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:27.350948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:27.351227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:27.351898 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:27.353254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6f0fb-68f1-4309-87d0-333f79a87e8f None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:27.359929 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 652/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jul 27 06:36:27 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:36:29.349242 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6786ae-1171-40ff-aa29-f7000f56b2d4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:29.349807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6786ae-1171-40ff-aa29-f7000f56b2d4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:29.350004 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6786ae-1171-40ff-aa29-f7000f56b2d4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:29.351380 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 650/1302] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:36:29.356359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d623-0376-4739-a39f-868266704f0e None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:29.356771 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d623-0376-4739-a39f-868266704f0e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:29.356924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d623-0376-4739-a39f-868266704f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:29.374841 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-49d1d623-0376-4739-a39f-868266704f0e None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:36:29.376022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-49d1d623-0376-4739-a39f-868266704f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:36:29.447366 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 653/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:36:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 06:36:29.469975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6786ae-1171-40ff-aa29-f7000f56b2d4 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:29.561912 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:29.569519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6786ae-1171-40ff-aa29-f7000f56b2d4 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:29.601464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:29.602060 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:29.602339 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:29.602610 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:29.603228 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:29.604623 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 06:36:29.613456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 06:36:29.615162 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application with Lock( Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self._enter() Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application return self.creator() Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application created_value = creator( Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: ERROR keystone.server.flask.application Jul 27 06:36:29.616423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 651/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jul 27 06:36:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 06:36:29.621172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1d623-0376-4739-a39f-868266704f0e None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:29.697213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:29.719230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1d623-0376-4739-a39f-868266704f0e None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:29.752178 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:29.752788 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:29.753073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:29.753352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] PATH_INFO: `/v3/services` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:29.754049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:36:29.754360 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:29.756440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac119e3-43e4-429b-9751-f563137d3092 None admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:29.761409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 654/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jul 27 06:36:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:36:29.767184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:36:29.866084 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87aeb0-1699-47cd-8760-cd0f8e1993d5 None admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:36:29.898792 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:36:29.899394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:36:29.899676 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:36:29.899953 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:36:29.900650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:36:29.902020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0920e81-0a7f-4f3f-92c7-a39fa021e742 None admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:36:29.921954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 652/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:36:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:36:39.758484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=30 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:36:39.917684 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:32.466298 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25387a-41d6-480f-beaf-d9b7e4720c11 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:32.467184 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25387a-41d6-480f-beaf-d9b7e4720c11 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:32.467621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25387a-41d6-480f-beaf-d9b7e4720c11 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:32.469417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 655/1307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:32.474981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d444c5-9f5b-4723-a53a-96f84bd709d9 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:32.475455 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d444c5-9f5b-4723-a53a-96f84bd709d9 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:32.475563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d444c5-9f5b-4723-a53a-96f84bd709d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:32.571111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c6d444c5-9f5b-4723-a53a-96f84bd709d9 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:32.572484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c6d444c5-9f5b-4723-a53a-96f84bd709d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:32.715966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 653/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:32.778444 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:32.778989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:32.779431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:32.780919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 656/1309] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 06:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:32.794820 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c835b9-de13-4039-bdd9-e1c49e827ec5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:32.795118 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c835b9-de13-4039-bdd9-e1c49e827ec5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:32.795400 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c835b9-de13-4039-bdd9-e1c49e827ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:32.900973 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-26c835b9-de13-4039-bdd9-e1c49e827ec5 None None] MFA Rules not processed for user `cb171d91fc464ac1ae2ade235f5c7608`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:32.903021 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-26c835b9-de13-4039-bdd9-e1c49e827ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:32.933354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:33.034396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 654/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 06:37:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:33.035986 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:33.039144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:33.194815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0c73c-172e-4f9b-89a5-dccad2f309f8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:33.282254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:33.283347 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:33.283779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:33.284200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:33.286907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:33.357089 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:33.422183 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:33.423619 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0558-a88a-4af9-a2fc-8b81c508c674 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:33.444618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 657/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:35.908826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:35.909355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:35.909558 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:35.911833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 655/1312] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:35.917354 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e78e-3f12-425d-b181-795904ae75fc None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:35.917766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e78e-3f12-425d-b181-795904ae75fc None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:35.917872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e78e-3f12-425d-b181-795904ae75fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:36.001302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-4fa2e78e-3f12-425d-b181-795904ae75fc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:36.002533 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4fa2e78e-3f12-425d-b181-795904ae75fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:36.099019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 658/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:36.137464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:36.139513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:36.250064 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93568d0-6a10-43b7-accd-a2d0d20d8df7 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:36.325303 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:36.328298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:36.329204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:36.329718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:36.330253 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:36.332597 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:36.444826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:36.446294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdd826-f470-451c-8a93-0ec402b09fda service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:36.465480 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 656/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:38.770856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372b3d-2e59-4b6e-b3da-564db07b0c66 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:38.771378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372b3d-2e59-4b6e-b3da-564db07b0c66 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:38.771471 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372b3d-2e59-4b6e-b3da-564db07b0c66 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:38.772768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 659/1315] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:38.777872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452998e-2775-4411-ba03-6de96ffb6457 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:38.778279 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452998e-2775-4411-ba03-6de96ffb6457 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:38.778451 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452998e-2775-4411-ba03-6de96ffb6457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:38.843210 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-7452998e-2775-4411-ba03-6de96ffb6457 None None] MFA Rules not processed for user `64192fa2adce46b7a1ac80ee290def63`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:38.844427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-7452998e-2775-4411-ba03-6de96ffb6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:38.895029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:38.939564 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 657/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:38.993059 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:38.993338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:38.993609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:38.995025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 660/1317] 10.222.0.36 () {54 vars in 965 bytes} [Wed Jul 27 06:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:39.000857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebf372-1ad6-4d58-b16a-78e6f3548138 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:39.001177 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebf372-1ad6-4d58-b16a-78e6f3548138 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:39.001457 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebf372-1ad6-4d58-b16a-78e6f3548138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:39.074415 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-38ebf372-1ad6-4d58-b16a-78e6f3548138 None None] MFA Rules not processed for user `cb171d91fc464ac1ae2ade235f5c7608`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:39.075619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-38ebf372-1ad6-4d58-b16a-78e6f3548138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:39.148736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 658/1318] 10.222.0.36 () {58 vars in 1062 bytes} [Wed Jul 27 06:37:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:39.157700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:39.159252 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:39.169495 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:39.262015 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df04ea-4021-450a-a618-57a8fcb1734b None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:39.334607 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:39.335221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:39.335503 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:39.335777 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:39.337372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:39.398696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:39.444705 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:39.446101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e735cd3-353b-4442-9401-64d0c7f3fd33 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:39.464271 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 661/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:42.404213 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:42.404750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:42.404750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:42.406172 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 659/1320] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:42.411296 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460e0a-6336-4fed-90d9-9d209abf2ec9 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:42.411726 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460e0a-6336-4fed-90d9-9d209abf2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:42.411868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460e0a-6336-4fed-90d9-9d209abf2ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:42.477079 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-43460e0a-6336-4fed-90d9-9d209abf2ec9 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:42.478310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-43460e0a-6336-4fed-90d9-9d209abf2ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:42.571336 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 662/1321] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:42.609701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:42.611512 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:42.662519 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:42.713187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2e41b-46c4-424d-9e08-bb31a6d92154 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:42.783466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:42.784071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:42.784350 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:42.785293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:42.786917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:42.878794 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:42.892399 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:42.893887 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729738a9-4c2d-4d8f-98b9-8744eacd4b0e service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:42.912018 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 660/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:45.601867 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:45.602431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:45.602431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:45.603798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 663/1323] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:45.609107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d94e-fcf1-435a-bea8-5a3033df78cc None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:45.609514 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d94e-fcf1-435a-bea8-5a3033df78cc None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:45.609659 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d94e-fcf1-435a-bea8-5a3033df78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:45.676633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-3678d94e-fcf1-435a-bea8-5a3033df78cc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:45.677830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3678d94e-fcf1-435a-bea8-5a3033df78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:45.775130 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 661/1324] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:45.813217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:45.814938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:45.836851 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:45.930102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaed30e-fe86-4658-b4d5-233d51f7bfb9 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:46.004781 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:46.005520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:46.005872 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:46.006292 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:46.007989 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:46.079008 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:46.117402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:46.118871 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff949acc-d64f-42cf-bc1e-c65f8c39bb95 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:46.136922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 664/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:48.212237 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:48.212798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:48.212798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:48.214319 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 662/1326] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:48.219249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465918-dc96-40d5-9874-8cbfe2727c01 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:48.219655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465918-dc96-40d5-9874-8cbfe2727c01 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:48.219934 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465918-dc96-40d5-9874-8cbfe2727c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:48.288814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-2b465918-dc96-40d5-9874-8cbfe2727c01 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:48.289921 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2b465918-dc96-40d5-9874-8cbfe2727c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:48.384810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 665/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:48.421259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:48.422827 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:48.483425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:48.526673 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7703-bc2e-4116-97a8-6c58bde4af9d None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:48.599298 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:48.599922 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:48.600205 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:48.600482 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:48.602138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:48.717775 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:48.720197 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:48.722641 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674863b6-5cf1-474a-b86d-e1ee040cb332 service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:48.742340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 663/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:50.787146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:50.787700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:50.787700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:50.789261 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 666/1329] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:50.794840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8092-ae4d-47c2-8ca9-74c344d7c4d4 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:50.795392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8092-ae4d-47c2-8ca9-74c344d7c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:50.795392 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8092-ae4d-47c2-8ca9-74c344d7c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:50.862943 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-98fa8092-ae4d-47c2-8ca9-74c344d7c4d4 None None] MFA Rules not processed for user `64192fa2adce46b7a1ac80ee290def63`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:50.864149 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-98fa8092-ae4d-47c2-8ca9-74c344d7c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:50.959830 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 664/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:50.996881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:50.998428 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:51.030291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:51.107401 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440b946-ada3-44e8-ab19-f04e8512af70 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:51.183030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:51.183653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:51.183933 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:51.184208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:51.185804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:51.262906 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:51.295731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:51.297167 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e439923-1993-4ef5-ad67-157a50e06315 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:51.316496 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 667/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:53.608868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:53.609460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:53.609460 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:53.610817 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 665/1332] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:53.616801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126149f-64ee-45d5-b5bd-392db1e509ba None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:53.617280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126149f-64ee-45d5-b5bd-392db1e509ba None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:53.617410 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126149f-64ee-45d5-b5bd-392db1e509ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:53.682611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-3126149f-64ee-45d5-b5bd-392db1e509ba None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:53.683826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3126149f-64ee-45d5-b5bd-392db1e509ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:53.783957 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 668/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:53.820627 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:53.822187 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:53.893556 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:53.928996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714b787-bed3-4d60-b85f-6911066fba51 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:54.002644 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:54.003266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:54.003549 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:54.003824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:54.005448 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:54.110878 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:54.113141 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a8109-831f-4f7e-90d2-6cabd1a1703e service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:54.122940 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:54.131501 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 666/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:56.193463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:56.194039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:56.194039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:56.195397 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 669/1335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:56.200450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b35e2-583e-4c48-9972-852ded4aaeea None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:56.200880 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b35e2-583e-4c48-9972-852ded4aaeea None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:56.201015 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b35e2-583e-4c48-9972-852ded4aaeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:56.266711 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-271b35e2-583e-4c48-9972-852ded4aaeea None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:56.267968 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-271b35e2-583e-4c48-9972-852ded4aaeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:56.367138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 667/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:56.407658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:56.409849 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:56.450251 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:56.513374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5834c3-2ae0-410f-9452-55a14b53d2ac None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:56.585572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:56.586220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:56.586505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:56.586783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:56.588406 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:56.672025 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:56.697548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:56.699024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b16d9e-345e-4b32-b5c1-4f23dcceb708 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:56.717768 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 670/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:37:59.048091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:59.048619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:59.048619 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:59.050635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 668/1338] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:37:59.055997 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3600c-989d-4184-952b-8059581bf3c8 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:59.056408 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3600c-989d-4184-952b-8059581bf3c8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:59.056551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3600c-989d-4184-952b-8059581bf3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:59.123302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-b1d3600c-989d-4184-952b-8059581bf3c8 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:37:59.124497 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b1d3600c-989d-4184-952b-8059581bf3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:59.222110 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 671/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:37:59.260632 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:37:59.262250 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:59.336128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:37:59.366774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5046b-684a-438e-9d81-48e04e2f7cbb None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:37:59.440091 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:37:59.440826 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:37:59.441103 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:37:59.441373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:37:59.443040 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:37:59.548840 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:37:59.550444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038a804-96a1-40c4-8fc9-d6f852fccf93 service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:37:59.567760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 669/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:37:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:03.716658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:03.717173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:03.717173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:03.718508 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 672/1341] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:03.724143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849198ef-12a6-4e3c-b818-2b774d99f19c None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:03.724606 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849198ef-12a6-4e3c-b818-2b774d99f19c None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:03.724828 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849198ef-12a6-4e3c-b818-2b774d99f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:03.730912 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:03.792456 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-849198ef-12a6-4e3c-b818-2b774d99f19c None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:03.793655 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-849198ef-12a6-4e3c-b818-2b774d99f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:03.887547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 670/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:03.926578 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:03.928203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:03.979665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:04.036404 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ab3d6-95d0-477d-ac26-09b64d20b861 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:04.113966 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:04.114653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:04.114939 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:04.115213 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:04.116819 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:04.211717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:04.229523 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:04.231049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d83f-ca6e-4608-80bb-c70481e7c12b service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:04.250102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 673/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:06.882321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:06.882890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:06.882890 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:06.884215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 671/1344] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:06.890052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8c8e-0686-45a2-9fb1-d67c21543918 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:06.890587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8c8e-0686-45a2-9fb1-d67c21543918 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:06.890587 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8c8e-0686-45a2-9fb1-d67c21543918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:06.962405 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-ca6c8c8e-0686-45a2-9fb1-d67c21543918 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:06.963797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ca6c8c8e-0686-45a2-9fb1-d67c21543918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:07.068510 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 674/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:07.106426 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:07.107933 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:07.194814 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:07.212712 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a4492-f867-4539-8aa0-baaa6253b945 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:07.286724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:07.287234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:07.287509 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:07.287777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:07.289387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:07.401159 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:07.402737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866fada-84e4-49e5-8bbb-bb811485261e service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:07.427012 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 672/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:11.666783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:11.667203 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:11.667351 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:11.668737 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 675/1347] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:11.674917 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74c6cc-8d9a-4ee7-90dc-c44cd3611e39 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:11.675403 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74c6cc-8d9a-4ee7-90dc-c44cd3611e39 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:11.675662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74c6cc-8d9a-4ee7-90dc-c44cd3611e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:11.692547 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:11.749447 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-0c74c6cc-8d9a-4ee7-90dc-c44cd3611e39 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:11.750704 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-0c74c6cc-8d9a-4ee7-90dc-c44cd3611e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:11.851918 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 673/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:11.891001 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:11.892525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:11.954478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:12.004825 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9040f9-fa94-4650-a9f3-9a94fb85ef1f None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:12.086294 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:12.086881 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:12.087161 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:12.087437 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:12.089103 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:12.199093 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:12.203166 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:12.204590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe12822-d40d-4f06-bed1-0554c92724c3 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:12.224476 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 676/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:17.179932 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433e-4015-48c3-82a8-430b71269354 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:17.180423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433e-4015-48c3-82a8-430b71269354 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:17.180423 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433e-4015-48c3-82a8-430b71269354 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:17.181996 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 674/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 06:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:17.187208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1730-6d42-4638-9c23-9e1e7e7d1908 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:17.187631 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1730-6d42-4638-9c23-9e1e7e7d1908 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:17.187762 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1730-6d42-4638-9c23-9e1e7e7d1908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:17.310218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-e49f1730-6d42-4638-9c23-9e1e7e7d1908 None None] MFA Rules not processed for user `5da5ce7121544c5f9838982de97b645c`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:17.311673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e49f1730-6d42-4638-9c23-9e1e7e7d1908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:17.436034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 677/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:21.406408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac9514-d18f-4641-af26-7b992e017580 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:21.406974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac9514-d18f-4641-af26-7b992e017580 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:21.406974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac9514-d18f-4641-af26-7b992e017580 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:21.408276 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 675/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 06:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:21.413618 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f54993-945f-4e39-8dbb-749bc578537e None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:21.414121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f54993-945f-4e39-8dbb-749bc578537e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:21.414515 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f54993-945f-4e39-8dbb-749bc578537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:21.476747 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:21.502199 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-79f54993-945f-4e39-8dbb-749bc578537e None None] MFA Rules not processed for user `5da5ce7121544c5f9838982de97b645c`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:21.504389 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-79f54993-945f-4e39-8dbb-749bc578537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:21.618803 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 678/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:21.849860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=59 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:23.242304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da668b-09d7-4f7a-a0dc-6d401379baf4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:23.243025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da668b-09d7-4f7a-a0dc-6d401379baf4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:23.243025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da668b-09d7-4f7a-a0dc-6d401379baf4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:23.244221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 676/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jul 27 06:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:23.249168 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440e14d-7228-44eb-a5c1-096898d35c37 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:23.249625 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440e14d-7228-44eb-a5c1-096898d35c37 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:23.250046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440e14d-7228-44eb-a5c1-096898d35c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:23.328313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-c440e14d-7228-44eb-a5c1-096898d35c37 None None] MFA Rules not processed for user `5da5ce7121544c5f9838982de97b645c`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:23.329514 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c440e14d-7228-44eb-a5c1-096898d35c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:23.371121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:23.437912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 679/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jul 27 06:38:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:38:26.794390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1698c-5096-41f9-9894-371237b72442 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:26.794913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1698c-5096-41f9-9894-371237b72442 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:26.794913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1698c-5096-41f9-9894-371237b72442 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:26.796390 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 677/1356] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:26.803108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5adf8-2000-40df-92e7-b5d4636859d3 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:26.803594 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5adf8-2000-40df-92e7-b5d4636859d3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:26.803692 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5adf8-2000-40df-92e7-b5d4636859d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:26.885187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-cba5adf8-2000-40df-92e7-b5d4636859d3 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:26.886481 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-cba5adf8-2000-40df-92e7-b5d4636859d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:26.993868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 680/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:27.044440 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:27.044725 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:27.045001 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:27.046383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 678/1358] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 06:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:27.055636 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6362adb-3b82-47fe-a28a-ff31ae782985 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:27.055938 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6362adb-3b82-47fe-a28a-ff31ae782985 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:27.056333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6362adb-3b82-47fe-a28a-ff31ae782985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:27.080092 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:27.147142 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-d6362adb-3b82-47fe-a28a-ff31ae782985 None None] MFA Rules not processed for user `b426ebd709e942819f19926f2e3668d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:27.148205 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-d6362adb-3b82-47fe-a28a-ff31ae782985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:27.274247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 681/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 06:38:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:27.282234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:27.284689 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:27.437702 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d953212-686c-4c2d-a925-39806b677b99 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:27.514385 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:27.558884 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:27.559506 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:27.559781 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:27.560053 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:27.561657 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:27.664222 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:27.665960 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb6f25-0893-4046-bd45-d07b4987648a service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:27.684970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 679/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:29.913445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:29.914063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:29.914063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:29.915575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 682/1361] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:29.920541 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a595-99f6-4054-a2c4-e27221694e15 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:29.921028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a595-99f6-4054-a2c4-e27221694e15 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:29.921193 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a595-99f6-4054-a2c4-e27221694e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:29.991081 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-2685a595-99f6-4054-a2c4-e27221694e15 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:29.992296 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-2685a595-99f6-4054-a2c4-e27221694e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:30.039183 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:30.102340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 680/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:30.146868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:30.148524 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:30.224962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:30.323076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108aca9-d1d0-4eca-b26b-98880c892878 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:30.465602 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:30.466344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:30.466687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:30.467022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:30.468808 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:30.614529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:30.694220 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:30.695734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e370c-bf78-4f96-be2c-9705b76e78a6 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:30.714041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 683/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:33.085245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb5d1-a036-420b-8e1b-4a090e28d23d None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:33.085683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb5d1-a036-420b-8e1b-4a090e28d23d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:33.085848 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb5d1-a036-420b-8e1b-4a090e28d23d None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:33.087199 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 681/1364] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:33.092685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5df54-35da-4a5a-b56d-0d6c728b86ba None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:33.093563 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5df54-35da-4a5a-b56d-0d6c728b86ba None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:33.094105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5df54-35da-4a5a-b56d-0d6c728b86ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:33.165028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-2ea5df54-35da-4a5a-b56d-0d6c728b86ba None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:33.166646 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2ea5df54-35da-4a5a-b56d-0d6c728b86ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:33.264571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:33.274344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 684/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:33.317620 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:33.318246 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:33.318772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:33.320599 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 682/1366] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jul 27 06:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:33.326945 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782b93-5970-4f03-9365-4d309dae1ef7 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:33.327367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782b93-5970-4f03-9365-4d309dae1ef7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:33.327658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2782b93-5970-4f03-9365-4d309dae1ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:33.412338 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-c2782b93-5970-4f03-9365-4d309dae1ef7 None None] MFA Rules not processed for user `b426ebd709e942819f19926f2e3668d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:33.413499 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c2782b93-5970-4f03-9365-4d309dae1ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:33.546774 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 685/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jul 27 06:38:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:33.555860 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:33.557520 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:33.734594 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c587567-2ba8-467f-9769-ee839823a6e8 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:33.757307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:33.864329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:33.864954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:33.865234 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:33.865683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:33.867471 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:33.972952 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:33.974761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d8005-cabd-4c70-84c6-9aa95268afd5 service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:33.996029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 683/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:34.874778 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dbabd-df69-4527-b554-9ae19da22974 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:34.875313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dbabd-df69-4527-b554-9ae19da22974 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:34.875313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dbabd-df69-4527-b554-9ae19da22974 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:34.876859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 686/1369] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:34.882062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77d96-d457-428e-849a-89dcd2f31ff0 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:34.882523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77d96-d457-428e-849a-89dcd2f31ff0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:34.882634 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77d96-d457-428e-849a-89dcd2f31ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:34.918635 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:34.954612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-11d77d96-d457-428e-849a-89dcd2f31ff0 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:34.956354 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-11d77d96-d457-428e-849a-89dcd2f31ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:35.068593 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 684/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:35.417454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:35.417454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:35.417454 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:35.418801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 687/1371] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 06:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:35.423591 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303486-2058-46fa-b46e-87c4d8084e7f None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:35.424643 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303486-2058-46fa-b46e-87c4d8084e7f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:35.424990 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303486-2058-46fa-b46e-87c4d8084e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:35.508454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c4303486-2058-46fa-b46e-87c4d8084e7f None None] MFA Rules not processed for user `b9f7fa9ae17a4ed7a717ed044eabca83`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:35.509798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c4303486-2058-46fa-b46e-87c4d8084e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:35.558572 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:35.593146 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 685/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 06:38:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:35.600956 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:35.602575 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:35.674610 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:35.730693 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b398e-5e8a-44ca-bfbf-e5776c746a24 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:35.807277 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f7fa9ae17a4ed7a717ed044eabca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:35.807893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:35.808176 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:35.808442 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:35.810063 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:35.916020 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:35.930962 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:35.932399 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f97db-1da7-4e84-8dfa-bf91921b78bd service cinder] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:35.953561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 688/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:36.093009 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:36.093491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:36.093756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:36.094970 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 686/1374] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:36.101026 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98206a-3c67-47f4-bd21-143bc9998efc None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:36.101316 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98206a-3c67-47f4-bd21-143bc9998efc None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:36.101656 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98206a-3c67-47f4-bd21-143bc9998efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:36.184561 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-2a98206a-3c67-47f4-bd21-143bc9998efc None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:36.186102 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2a98206a-3c67-47f4-bd21-143bc9998efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:36.318868 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 689/1375] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:36.363343 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:36.364868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:36.509843 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e7151-43fa-4b7f-97f3-8959b2f87734 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:36.560003 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:36.623975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:36.624605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:36.624895 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:36.625168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:36.626857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:36.721465 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:36.723089 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac5888-c584-4a4e-bdce-58fb1ac10e65 service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:36.741171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 687/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:38.779856 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:38.780385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:38.780385 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:38.781763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 690/1377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:38.787047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817f85-7096-4e26-981e-bdf7d5ea37c5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:38.787477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817f85-7096-4e26-981e-bdf7d5ea37c5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:38.787615 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817f85-7096-4e26-981e-bdf7d5ea37c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:38.860048 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ab817f85-7096-4e26-981e-bdf7d5ea37c5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:38.862404 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ab817f85-7096-4e26-981e-bdf7d5ea37c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:38.870171 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:38.990485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 688/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:39.034138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:39.035698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:39.095013 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:39.178518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080a24f-8ec8-47bf-af6e-af4cedeea454 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:39.302328 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:39.302963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:39.303236 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:39.303507 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:39.305129 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:39.328402 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:39.420592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:39.422144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe4e4c-9f36-4750-ab15-7328d9cf8125 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:39.445972 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 691/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:41.516777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:41.517321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:41.517321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:41.518733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 689/1380] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:41.524105 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aabff-0694-4c35-89bd-8e11ce77e14d None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:41.524519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aabff-0694-4c35-89bd-8e11ce77e14d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:41.524688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aabff-0694-4c35-89bd-8e11ce77e14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:41.594654 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-3f8aabff-0694-4c35-89bd-8e11ce77e14d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:41.595924 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3f8aabff-0694-4c35-89bd-8e11ce77e14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:41.700763 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:41.714069 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 692/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:41.754367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:41.756039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:41.904806 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:41.916621 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad80e2-dee8-42bb-83c1-3aa5cf00ba01 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:42.038224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:42.038858 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:42.039209 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:42.039553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:42.041235 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:42.138824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:42.140485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d171b-ebf8-44e9-859e-b04a66df7f19 service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:42.152669 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:42.162031 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 690/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:44.507041 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:44.507600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:44.507600 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:44.508936 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 693/1383] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:44.514274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6b2d-2ec9-4550-8ca3-79b640c6e686 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:44.514694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6b2d-2ec9-4550-8ca3-79b640c6e686 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:44.514816 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab6b2d-2ec9-4550-8ca3-79b640c6e686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:44.596277 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-b5ab6b2d-2ec9-4550-8ca3-79b640c6e686 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:44.597837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-b5ab6b2d-2ec9-4550-8ca3-79b640c6e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:44.716492 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 691/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:44.760280 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:44.761807 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:44.917633 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497692d-a62c-4ae5-9112-671e1508e96c None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:44.989128 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:45.036505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:45.037042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:45.037322 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:45.037595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:45.039257 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:45.131145 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:45.132743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64f508-de41-4106-aaf1-2bd9813cb45b service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:45.151104 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 694/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:47.190804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:47.191790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:47.191790 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:47.193019 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 692/1386] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:47.198664 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c2d29-3b50-4974-808a-7ffc185ab83e None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:47.199415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c2d29-3b50-4974-808a-7ffc185ab83e None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:47.199415 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c2d29-3b50-4974-808a-7ffc185ab83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:47.269072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-3b0c2d29-3b50-4974-808a-7ffc185ab83e None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:47.270334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3b0c2d29-3b50-4974-808a-7ffc185ab83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:47.309010 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:47.388459 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 695/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:47.434085 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:47.435612 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:47.516247 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:47.587579 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01291f3-d287-4874-b757-5fca6fa13667 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:47.711753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:47.718617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:47.718617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:47.718617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:47.718617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:47.754819 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:47.844245 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:47.848073 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca987ffd-610b-42bf-9748-71c743e6b4c0 service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:47.883287 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 693/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:49.967022 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:49.967590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:49.967590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:49.968854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 696/1389] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:49.974334 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7d9-348f-40d3-8f2c-5de891aee11b None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:49.974807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7d9-348f-40d3-8f2c-5de891aee11b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:49.974914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7d9-348f-40d3-8f2c-5de891aee11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:50.041496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-f9f5a7d9-348f-40d3-8f2c-5de891aee11b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:50.042721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f9f5a7d9-348f-40d3-8f2c-5de891aee11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:50.142981 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 694/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:50.187144 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:50.188704 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:50.334172 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb0c18-0311-42eb-ace3-abd4bf872d25 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:50.350686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:50.448550 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:50.449168 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:50.449443 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:50.449720 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:50.451409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:50.540703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:50.542453 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c331b-53f0-4aa3-a6fc-ee155e222503 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:50.559982 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 697/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:50.984371 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abfe7c-36f6-4dda-b248-74abe38c6cc4 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:50.984896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abfe7c-36f6-4dda-b248-74abe38c6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:50.984896 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abfe7c-36f6-4dda-b248-74abe38c6cc4 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:50.986835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 695/1392] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:38:50.992227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45079-a4e0-4b41-9a42-aee4a7b7e2ff None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:50.992662 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45079-a4e0-4b41-9a42-aee4a7b7e2ff None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:50.992847 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45079-a4e0-4b41-9a42-aee4a7b7e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:51.018072 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:51.059678 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-e2c45079-a4e0-4b41-9a42-aee4a7b7e2ff None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:51.060874 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e2c45079-a4e0-4b41-9a42-aee4a7b7e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:51.155452 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 698/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:51.213502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:51.213793 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:51.214108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:51.215478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 696/1394] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 06:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:51.221016 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4e64-1d8a-4302-a385-2a61bb4d90f6 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:51.221300 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4e64-1d8a-4302-a385-2a61bb4d90f6 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:51.221576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb4e64-1d8a-4302-a385-2a61bb4d90f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:51.297963 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-7cbb4e64-1d8a-4302-a385-2a61bb4d90f6 None None] MFA Rules not processed for user `665904332c794638bd3a890fe602cfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:51.299440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-7cbb4e64-1d8a-4302-a385-2a61bb4d90f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:51.339145 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:51.375249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 699/1395] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 06:38:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:51.382718 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:51.384225 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:51.398069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:51.485748 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fbc9b-980b-4733-b728-197f67f58198 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:51.558945 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '665904332c794638bd3a890fe602cfb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:51.559483 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:51.559759 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:51.560046 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:51.561617 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:51.629256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:51.671967 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:51.673408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e3312-162e-428b-944a-dee3b1d3acd7 service glance] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:51.692546 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 697/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:52.428860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151d4e6-d8f7-4965-ab66-a10ff523442a None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:52.429363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151d4e6-d8f7-4965-ab66-a10ff523442a None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:52.429363 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151d4e6-d8f7-4965-ab66-a10ff523442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:52.495520 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9151d4e6-d8f7-4965-ab66-a10ff523442a None None] MFA Rules not processed for user `793cd10dcc4344f2be80fd34a82c0d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:52.496729 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9151d4e6-d8f7-4965-ab66-a10ff523442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:52.581189 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 700/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jul 27 06:38:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:52.625398 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdc80-87ab-411f-9b63-e9eee3dc42f6 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:52.625881 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdc80-87ab-411f-9b63-e9eee3dc42f6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:52.626028 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdc80-87ab-411f-9b63-e9eee3dc42f6 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:52.628086 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 698/1398] 10.222.0.36 () {54 vars in 943 bytes} [Wed Jul 27 06:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:52.634239 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:52.634530 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:52.634661 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b6a05-0c84-4de7-a087-ace126556fd8 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:52.634810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:52.635065 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b6a05-0c84-4de7-a087-ace126556fd8 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:52.635459 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b6a05-0c84-4de7-a087-ace126556fd8 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:52.636804 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 699/1399] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:38:52.641975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:52.642984 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:52.643329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:52.712571 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:52.714239 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:52.715796 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:52.721557 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] MFA Rules not processed for user `4a1b7d1e6073474081c39f043ac1ba26`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:52.722902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:52.813835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 700/1400] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:38:52.828821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 701/1401] 10.222.0.36 () {58 vars in 1040 bytes} [Wed Jul 27 06:38:52 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:52.838844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:52.840155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:52.858685 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:52.860565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:52.866836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:52.946155 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7bdd-805d-4918-9965-d6002f712a00 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:52.964576 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cdfc2-2967-4e04-8b89-71168ed2ce62 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:52.986595 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:53.055120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a1b7d1e6073474081c39f043ac1ba26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:53.055761 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:53.056039 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:53.056314 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:53.057994 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:53.083789 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:53.084398 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:53.084675 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:53.084944 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:53.086564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:53.100770 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:53.143918 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:53.145498 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c30bf-b554-49f2-9a75-b5facd95608e service swift] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:53.153435 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 701/1402] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jul 27 06:38:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 06:38:53.183333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:53.185559 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca80ea1-956f-458f-b531-ef66715fc8e3 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:53.205880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 702/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 06:38:55.304168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:55.304710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:55.304710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:55.306116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 702/1404] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:38:55.311422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031f7c6-d11c-4936-a108-4deae3bb3879 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:55.311909 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031f7c6-d11c-4936-a108-4deae3bb3879 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:55.312014 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031f7c6-d11c-4936-a108-4deae3bb3879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:55.347101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:55.387162 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6031f7c6-d11c-4936-a108-4deae3bb3879 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:55.388386 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6031f7c6-d11c-4936-a108-4deae3bb3879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:55.493422 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 703/1405] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:55.539256 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:55.541036 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:55.698286 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaa682-9581-4fdd-a2de-9727df61a2f6 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:55.739331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:55.835746 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:55.836367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:55.836648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:55.836923 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:55.838576 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:55.934802 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:55.936428 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87dbea-9f2e-4781-aded-1e24cc789b5c service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:55.955853 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 703/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:38:58.080809 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:58.081357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:58.081357 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:58.087169 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 704/1407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:38:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:38:58.088842 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1be7-d6e6-44c3-93e7-326472637ace None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:58.089367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1be7-d6e6-44c3-93e7-326472637ace None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:58.089470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1be7-d6e6-44c3-93e7-326472637ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:58.144513 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:58.171454 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-10fc1be7-d6e6-44c3-93e7-326472637ace None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:38:58.172687 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-10fc1be7-d6e6-44c3-93e7-326472637ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:58.291377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 704/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:38:58.357395 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:38:58.359356 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:58.480053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:58.541374 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce1da-788f-4183-baee-51eb5a34239c None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:38:58.687486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:38:58.688101 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:38:58.688375 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:38:58.688644 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:38:58.691062 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:38:58.768554 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:38:58.790861 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:38:58.792486 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5f0b9-2534-4926-8e3c-881b5dc4d99b service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:38:58.816418 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 705/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:08.285929 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=47 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:08.812445 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=16 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:15.989886 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2140-617e-4de0-8184-185a54db5f21 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:15.990445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2140-617e-4de0-8184-185a54db5f21 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:15.990445 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2140-617e-4de0-8184-185a54db5f21 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:15.991752 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 705/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 06:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:15.997083 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3473b4-ae15-4fb1-ab2f-260f7b4b01db None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:15.997504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3473b4-ae15-4fb1-ab2f-260f7b4b01db None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:15.997640 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3473b4-ae15-4fb1-ab2f-260f7b4b01db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:16.073400 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9f3473b4-ae15-4fb1-ab2f-260f7b4b01db None None] MFA Rules not processed for user `5da5ce7121544c5f9838982de97b645c`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:16.074634 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9f3473b4-ae15-4fb1-ab2f-260f7b4b01db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:16.190784 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 706/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 06:39:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:16.206168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794f38-8424-457f-bc5c-57522b599030 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:16.206464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794f38-8424-457f-bc5c-57522b599030 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:16.206743 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794f38-8424-457f-bc5c-57522b599030 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:16.208142 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 706/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jul 27 06:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:16.212734 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba318-c36e-451e-bb86-3270522c32c4 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:16.213049 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba318-c36e-451e-bb86-3270522c32c4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:16.213323 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba318-c36e-451e-bb86-3270522c32c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:16.284800 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:16.287231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6d4ba318-c36e-451e-bb86-3270522c32c4 None None] MFA Rules not processed for user `b426ebd709e942819f19926f2e3668d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:16.289030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6d4ba318-c36e-451e-bb86-3270522c32c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:16.403076 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 707/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jul 27 06:39:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:18.864823 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:18.865381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:18.865381 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:18.866733 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:18.872055 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21975ff-7043-493d-9778-285a03c8d739 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:18.872483 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21975ff-7043-493d-9778-285a03c8d739 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:18.872605 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21975ff-7043-493d-9778-285a03c8d739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:18.938463 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-f21975ff-7043-493d-9778-285a03c8d739 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:18.940339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f21975ff-7043-493d-9778-285a03c8d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:18.995812 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:19.037655 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:19.080976 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:19.082555 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:19.224667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cc063-13cc-42d3-9fdf-454bc36120b3 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:19.294675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:19.334948 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:19.335567 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:19.335844 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:19.336116 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:19.337720 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:19.425034 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:19.426694 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159855df-ab98-4888-860b-0748e434e8fb service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:19.444294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 708/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:20.339544 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:20.340122 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:20.340122 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:20.341494 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:20.346322 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13daa84-027c-4c5e-bda1-783374d6bf04 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:20.346736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13daa84-027c-4c5e-bda1-783374d6bf04 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:20.346852 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13daa84-027c-4c5e-bda1-783374d6bf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:20.421108 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d13daa84-027c-4c5e-bda1-783374d6bf04 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:20.422340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d13daa84-027c-4c5e-bda1-783374d6bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:20.449776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:20.519143 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:20.670805 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:20.672309 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:20.818993 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefb898-10fa-4ed3-ad62-e3e08eb1f142 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:20.858468 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:20.932795 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:20.933330 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:20.933609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:20.933883 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:20.935525 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:21.025250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:21.026859 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e639dde-d33c-4496-a0e1-9925c149b1ab service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:21.044810 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 710/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:21.925813 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:21.926406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:21.926406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:21.927683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:21.932829 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830cdc-d5bd-42db-8709-05fabc5b9184 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:21.933267 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830cdc-d5bd-42db-8709-05fabc5b9184 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:21.933367 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830cdc-d5bd-42db-8709-05fabc5b9184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:21.989121 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:22.011684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-1e830cdc-d5bd-42db-8709-05fabc5b9184 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:22.012891 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-1e830cdc-d5bd-42db-8709-05fabc5b9184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:22.110186 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:22.262839 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:22.264268 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:22.405340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ae42d-efa0-41d8-a049-b84e93c15f04 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:22.417671 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:22.516539 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:22.517088 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:22.517364 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:22.517634 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:22.519266 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:22.606496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:22.608070 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61294a7e-049e-4471-9eb0-49e2e3b4dccf service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:22.626356 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 711/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:23.483493 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:23.484084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:23.484084 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:23.485413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:23.490515 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c041-f53f-4d86-9603-bdc740162597 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:23.490997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c041-f53f-4d86-9603-bdc740162597 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:23.491105 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c041-f53f-4d86-9603-bdc740162597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:23.510582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:23.566198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-67e5c041-f53f-4d86-9603-bdc740162597 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:23.567406 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-67e5c041-f53f-4d86-9603-bdc740162597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:23.663487 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:23.816359 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:23.817823 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:23.928503 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:23.964324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519e945-ef38-490c-8ae9-57b6dd57b2b4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:24.073372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:24.073901 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:24.074204 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:24.074475 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:24.076061 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:24.159797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:24.173984 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:24.175565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22fe9f-da47-457f-8de0-9973d69e8bb2 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:24.193860 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 713/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:25.053914 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:25.054491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:25.054491 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:25.055777 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:25.060699 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78edc3-6cb3-44bd-996a-4a1c2d0763e6 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:25.061150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78edc3-6cb3-44bd-996a-4a1c2d0763e6 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:25.061259 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78edc3-6cb3-44bd-996a-4a1c2d0763e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:25.135968 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-af78edc3-6cb3-44bd-996a-4a1c2d0763e6 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:25.137160 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-af78edc3-6cb3-44bd-996a-4a1c2d0763e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:25.230043 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:25.385462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:25.386986 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:25.468653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:25.529951 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4228-2237-4845-bd3f-209d96ece90b None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:25.640396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:25.640946 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:25.641231 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:25.641574 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:25.643710 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:25.691622 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:25.732937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:25.734605 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedae8a-b781-4b3a-b222-a58f7a505e2c service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:25.754531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 714/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:26.553432 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:26.554009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:26.554009 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:26.555409 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:26.560178 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df7e9-86af-4be0-b55f-d0d447dc9300 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:26.560637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df7e9-86af-4be0-b55f-d0d447dc9300 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:26.560749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df7e9-86af-4be0-b55f-d0d447dc9300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:26.635583 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-e72df7e9-86af-4be0-b55f-d0d447dc9300 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:26.636784 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e72df7e9-86af-4be0-b55f-d0d447dc9300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:26.735198 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:26.887249 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:26.888760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:26.937981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:27.034158 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777879b-29e2-4e7e-a033-57e492fd2ee2 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:27.144815 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:27.145352 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:27.145632 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:27.145907 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:27.147529 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:27.154288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:27.242315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:27.243918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f30cf-da8c-4704-a91d-fdf1404f6b84 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:27.261760 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 716/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:28.064835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:28.065357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:28.065484 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:28.066919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:28.071821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac24c-ab1b-4153-8c64-00e217f8d60f None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:28.072233 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac24c-ab1b-4153-8c64-00e217f8d60f None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:28.072378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac24c-ab1b-4153-8c64-00e217f8d60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:28.155064 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9f0ac24c-ab1b-4153-8c64-00e217f8d60f None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:28.156293 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9f0ac24c-ab1b-4153-8c64-00e217f8d60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:28.223764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:28.252801 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:28.403937 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:28.405468 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:28.423531 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:28.549119 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea66a5-55ed-401c-83ea-f3185b952540 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:28.632667 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:28.659323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:28.659870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:28.660148 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:28.660418 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:28.662050 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:28.748919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:28.750524 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a08932-3092-4e56-abdf-379b258b922f service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:28.769150 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 717/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:29.582513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:29.583057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:29.583057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:29.584424 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:29.589340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d058-119e-48c3-8d71-6d42aed96d8e None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:29.589771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d058-119e-48c3-8d71-6d42aed96d8e None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:29.589882 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d058-119e-48c3-8d71-6d42aed96d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:29.665697 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-2156d058-119e-48c3-8d71-6d42aed96d8e None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:29.666931 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-2156d058-119e-48c3-8d71-6d42aed96d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:29.706831 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:29.762077 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:29.916687 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:29.918173 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:30.062053 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d955-1533-46d4-bfd7-f4495dcafd61 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:30.116217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:30.182380 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:30.182923 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:30.183211 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:30.183494 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:30.185119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:30.274912 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:30.276532 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dd3f9-d06a-4426-8bcb-db57d02eb562 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:30.294039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 719/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:31.093965 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:31.094494 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:31.094594 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:31.096259 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 06:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:31.101217 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601357-272c-480f-9586-01716b7c5991 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:31.101623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601357-272c-480f-9586-01716b7c5991 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:31.101772 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601357-272c-480f-9586-01716b7c5991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:31.177046 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-10601357-272c-480f-9586-01716b7c5991 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:31.178288 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-10601357-272c-480f-9586-01716b7c5991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:31.180519 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:31.273645 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 06:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:31.424357 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:31.426006 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:31.567485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfbb62-e79c-4131-b3ea-d4b497556229 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:31.593271 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:31.681002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:31.681628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:31.681912 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:31.682215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:31.683833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:31.773139 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:31.774767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e318-35ea-40f8-84f8-a394dd34ceeb service nova] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:31.793255 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 720/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:33.703790 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:33.704947 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:33.705334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:33.706739 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:33.711904 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb4b7-2fb3-4f16-9c7b-62a737191f9d None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:33.712321 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb4b7-2fb3-4f16-9c7b-62a737191f9d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:33.712466 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbb4b7-2fb3-4f16-9c7b-62a737191f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:33.754157 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:33.780020 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-43bbb4b7-2fb3-4f16-9c7b-62a737191f9d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:33.781241 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-43bbb4b7-2fb3-4f16-9c7b-62a737191f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:33.874966 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:33.914539 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:33.916032 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:34.018057 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed482-eb8b-4451-9726-30fd592bb738 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:34.049182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:34.090393 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:34.091011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:34.091291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:34.091564 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:34.093231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:34.209065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:34.210440 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a027d-a5b5-42f6-ba8d-e46f01a94d94 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:34.228674 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 722/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:36.380675 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644026-0275-467f-af3c-93f03b303499 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:36.381283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644026-0275-467f-af3c-93f03b303499 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:36.381283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644026-0275-467f-af3c-93f03b303499 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:36.382601 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:36.387727 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638ad0-c6d2-4545-9154-cd8045f6cb13 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:36.388146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638ad0-c6d2-4545-9154-cd8045f6cb13 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:36.388260 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638ad0-c6d2-4545-9154-cd8045f6cb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:36.454718 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-47638ad0-c6d2-4545-9154-cd8045f6cb13 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:36.455946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-47638ad0-c6d2-4545-9154-cd8045f6cb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:36.468378 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:36.550642 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:36.589107 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644026-0275-467f-af3c-93f03b303499 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:36.590708 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a7644026-0275-467f-af3c-93f03b303499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:36.696417 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644026-0275-467f-af3c-93f03b303499 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:36.709307 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:36.771535 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:36.772182 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:36.772464 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:36.772737 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:36.774394 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:36.880736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:36.882180 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa20014-64f4-4cf2-9d2f-738cd2e8fc30 service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:36.900280 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 723/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:39.200154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:39.200744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:39.200744 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:39.202099 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:39.207355 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edeffd-d575-470e-ad3c-3aece45bff1d None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:39.207807 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edeffd-d575-470e-ad3c-3aece45bff1d None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:39.207925 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edeffd-d575-470e-ad3c-3aece45bff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:39.257479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:39.275278 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-51edeffd-d575-470e-ad3c-3aece45bff1d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:39.276481 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-51edeffd-d575-470e-ad3c-3aece45bff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:39.370240 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:39.409817 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:39.411365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:39.514484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2080-1ff0-41cb-a5e4-7d60554435b4 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:39.554505 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:39.589108 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:39.589721 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:39.590034 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:39.590311 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:39.591896 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:39.705052 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:39.706518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898e107-1d57-451b-8587-492d01518087 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:39.726042 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 725/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:41.854341 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:41.854899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:41.854899 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:41.856274 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:41.861670 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0566-8084-487d-a54e-7aa6324825c9 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:41.862140 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0566-8084-487d-a54e-7aa6324825c9 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:41.862248 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0566-8084-487d-a54e-7aa6324825c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:41.927536 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-6b0b0566-8084-487d-a54e-7aa6324825c9 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:41.928676 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6b0b0566-8084-487d-a54e-7aa6324825c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:41.950003 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:42.026688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:42.064696 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:42.066327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:42.167450 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7e63-07f8-45e4-888c-af033ba2c0db None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:42.187427 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:42.241868 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:42.242496 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:42.242774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:42.243047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:42.244649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:42.349315 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:42.350762 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08567dbf-be0a-409a-8023-946b5be3b8dc service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:42.368936 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 726/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:44.414521 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:44.415095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:44.415095 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:44.416465 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 06:39:44.421731 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa7c7-37c6-4805-b7d0-cb9aabe79d54 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:44.422147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa7c7-37c6-4805-b7d0-cb9aabe79d54 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:44.422331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaa7c7-37c6-4805-b7d0-cb9aabe79d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:44.484782 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:44.487954 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-76eaa7c7-37c6-4805-b7d0-cb9aabe79d54 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:44.490047 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-76eaa7c7-37c6-4805-b7d0-cb9aabe79d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:44.583821 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:44.622369 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:44.623880 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:44.730254 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041c22d-b4b7-4e32-b31e-59f23f6ee8f5 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:44.780154 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:44.805030 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:44.805639 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:44.805915 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:44.806218 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:44.807814 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:44.914333 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:44.915758 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f9b0f-9edb-48af-bb88-ba6cbbe99ef3 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:44.933509 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 728/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:46.993872 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:46.994553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:46.994553 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:46.995745 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:47.002024 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80d1b-9c5c-4b8d-8526-bf8db307fca7 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:47.002478 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80d1b-9c5c-4b8d-8526-bf8db307fca7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:47.002588 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80d1b-9c5c-4b8d-8526-bf8db307fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:47.070840 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-83b80d1b-9c5c-4b8d-8526-bf8db307fca7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:47.072058 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-83b80d1b-9c5c-4b8d-8526-bf8db307fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:47.101056 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:47.169384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:47.208653 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:47.210227 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:47.315180 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91ecc3-8c63-441a-bea3-d4e57905289e None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:47.343329 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:47.388147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:47.388763 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:47.389045 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:47.389323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:47.390974 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:47.500138 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:47.502323 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d1bfe-2ad8-48ab-8c22-35acd33440b2 service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:47.519870 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 729/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 06:39:49.586065 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:49.586658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:49.586658 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:49.587948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:49.593122 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399dc9-6ea5-4844-85e6-3b80ef6f8911 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:49.593523 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399dc9-6ea5-4844-85e6-3b80ef6f8911 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:49.593716 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399dc9-6ea5-4844-85e6-3b80ef6f8911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:49.661098 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-c9399dc9-6ea5-4844-85e6-3b80ef6f8911 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:49.662383 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-c9399dc9-6ea5-4844-85e6-3b80ef6f8911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:49.670833 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:49.758760 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:49.797262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:49.798821 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:49.900291 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569c143-a4a2-473b-8a14-93a7d9f16212 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:49.954306 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:49.972854 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:49.973460 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:49.973736 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:49.974040 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:49.975635 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:50.081447 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:50.082918 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074306cb-ace0-4292-8dc0-ef720d49ce12 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:50.101573 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 731/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:52.151461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:52.152029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:52.152029 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:52.153387 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:52.159295 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90bbcd-2543-4f31-a8cd-3edad7ddefbf None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:52.159703 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90bbcd-2543-4f31-a8cd-3edad7ddefbf None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:52.159850 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90bbcd-2543-4f31-a8cd-3edad7ddefbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:52.227136 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9b90bbcd-2543-4f31-a8cd-3edad7ddefbf None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:52.234764 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9b90bbcd-2543-4f31-a8cd-3edad7ddefbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:52.273190 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:52.331798 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:52.369358 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:52.370956 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:52.473799 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40cc6b-8433-4244-a0e0-c9c6eb7631df None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:52.511078 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:52.546408 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:52.547017 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:52.547294 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:52.547609 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:52.549312 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:52.655648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:52.657094 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca399f6d-e313-4153-b4ae-36814e2bb09b service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:52.675613 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 732/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:54.681766 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:54.682270 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:54.682376 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:54.683725 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:54.689135 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4742-2a20-4622-8479-26dd6edac1c4 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:54.689568 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4742-2a20-4622-8479-26dd6edac1c4 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:54.689699 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4742-2a20-4622-8479-26dd6edac1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:54.756335 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-880f4742-2a20-4622-8479-26dd6edac1c4 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:54.757544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-880f4742-2a20-4622-8479-26dd6edac1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:54.775336 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:54.854600 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:39:54.892302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:54.893838 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:54.996134 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8454464-2b78-4640-88c8-96598b807d02 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:55.061698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:55.068698 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:55.069317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:55.069603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:55.069882 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:55.071528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:55.178547 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:55.179980 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa859f7-e09d-41d5-854a-a8d970bfa950 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:55.198080 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 734/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:39:57.312436 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:57.312982 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:57.312982 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:57.314879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 06:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:39:57.320283 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233af959-1f25-4c73-9c39-ee4c3517bfa3 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:57.320700 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233af959-1f25-4c73-9c39-ee4c3517bfa3 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:57.320830 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233af959-1f25-4c73-9c39-ee4c3517bfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:57.386413 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-233af959-1f25-4c73-9c39-ee4c3517bfa3 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:39:57.387611 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-233af959-1f25-4c73-9c39-ee4c3517bfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:57.439543 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:57.489157 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 06:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 06:39:57.526650 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:39:57.528186 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:57.631221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c36fa-ba1f-47c9-9004-c5ef5da1e65a None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:39:57.678648 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:39:57.705425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:39:57.706071 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:39:57.706374 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:39:57.706649 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:39:57.708288 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:39:57.813654 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:39:57.815204 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b7c89-8fd6-4e54-b41f-faf0886c54b2 service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:39:57.833257 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 735/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:07.487193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=22 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:07.831025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=66 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:17.308974 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050be9d-b849-4c36-a2bc-b076f238decf None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:17.309590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050be9d-b849-4c36-a2bc-b076f238decf None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:17.309590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050be9d-b849-4c36-a2bc-b076f238decf None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:17.310942 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 736/1471] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:17.316774 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5e0f-cdab-40db-9f9b-01b029e00a80 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:17.317179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5e0f-cdab-40db-9f9b-01b029e00a80 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:17.317331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5e0f-cdab-40db-9f9b-01b029e00a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:17.381258 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-83ee5e0f-cdab-40db-9f9b-01b029e00a80 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:17.382478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-83ee5e0f-cdab-40db-9f9b-01b029e00a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:17.478248 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 736/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:17.560208 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:17.560521 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:17.560826 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:17.562198 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 737/1473] 10.222.0.36 () {54 vars in 966 bytes} [Wed Jul 27 06:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:17.567589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837119c-6777-4613-9e32-49bb8c51410e None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:17.567879 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837119c-6777-4613-9e32-49bb8c51410e None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:17.568156 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837119c-6777-4613-9e32-49bb8c51410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:17.641473 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-a837119c-6777-4613-9e32-49bb8c51410e None None] MFA Rules not processed for user `665904332c794638bd3a890fe602cfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:17.642700 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-a837119c-6777-4613-9e32-49bb8c51410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:17.663022 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:17.718170 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 737/1474] 10.222.0.36 () {58 vars in 1063 bytes} [Wed Jul 27 06:40:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:17.725696 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:17.727241 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:17.829603 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198a4e-1c78-4496-96ee-293e268f0771 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:17.901731 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '665904332c794638bd3a890fe602cfb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:17.902384 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:17.902675 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:17.902948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:17.904570 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:17.958146 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:18.013017 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:18.014548 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b000bbf-fa6d-4db3-8211-f1b28b27f8f2 service glance] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:18.033313 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 738/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:20.123375 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:20.123944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:20.123944 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:20.125327 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 738/1476] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:20.130679 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a2d4-c3b5-4fe0-b823-04028675ce7d None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:20.131100 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a2d4-c3b5-4fe0-b823-04028675ce7d None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:20.131235 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a2d4-c3b5-4fe0-b823-04028675ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:20.197255 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-0033a2d4-c3b5-4fe0-b823-04028675ce7d None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:20.198490 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-0033a2d4-c3b5-4fe0-b823-04028675ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:20.303806 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 739/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:20.326373 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:20.327924 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:20.454133 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f75da-6b01-4249-bda3-5482f3e70c4b None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:20.501836 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:20.546384 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:20.546999 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:20.547396 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:20.547668 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:20.548353 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:20.550168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING py.warnings [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:40:20.550168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: warnings.warn(msg) Jul 27 06:40:20.550168 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: Jul 27 06:40:20.551310 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb342-98ac-43d5-b191-663d31681c67 admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:20.570111 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 739/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jul 27 06:40:20 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:40:22.585202 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:22.585783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:22.585783 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:22.587187 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 740/1479] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:22.592367 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350a08-46e3-4511-bfdd-a58a6daa6605 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:22.592767 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350a08-46e3-4511-bfdd-a58a6daa6605 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:22.592928 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1350a08-46e3-4511-bfdd-a58a6daa6605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:22.781184 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-d1350a08-46e3-4511-bfdd-a58a6daa6605 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:22.783072 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d1350a08-46e3-4511-bfdd-a58a6daa6605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:22.876477 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 740/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:22.921112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:22.923200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:22.933185 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:23.068890 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee56b-df8d-4cd7-9eac-924e29feedba None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:23.147835 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:23.180620 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b426ebd709e942819f19926f2e3668d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:23.181231 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:23.181508 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:23.181779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:23.183480 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:23.273011 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:23.274623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68be52-9db3-4ff2-8b6e-6e2a708b5860 service nova] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:23.292324 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 741/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:25.478701 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:25.479269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:25.479269 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:25.480663 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 741/1482] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:25.486215 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107a4-6ad1-484b-9171-82d7490ed01b None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:25.486653 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107a4-6ad1-484b-9171-82d7490ed01b None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:25.486761 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107a4-6ad1-484b-9171-82d7490ed01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:25.551717 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-9f6107a4-6ad1-484b-9171-82d7490ed01b None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:25.552922 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-9f6107a4-6ad1-484b-9171-82d7490ed01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:25.598663 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:25.648317 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 742/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:25.684563 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:25.686131 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:25.709295 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:25.787224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99499f8-7c62-47c0-9eca-0ae02821698f None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:25.858283 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:25.858900 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:25.859176 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:25.859446 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:25.861630 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:25.932215 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:25.967083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:25.968412 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470f17-1627-4546-a650-86bde34cb0cf service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:25.986658 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 742/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:28.024621 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:28.025150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:28.025150 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:28.026484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 743/1485] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:28.031744 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcd9f-d4db-4b9e-871e-3bd7b56676ef None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:28.032331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcd9f-d4db-4b9e-871e-3bd7b56676ef None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:28.032331 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcd9f-d4db-4b9e-871e-3bd7b56676ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:28.097603 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-e39fcd9f-d4db-4b9e-871e-3bd7b56676ef None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:28.098837 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e39fcd9f-d4db-4b9e-871e-3bd7b56676ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:28.193340 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 743/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:28.231477 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:28.233037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:28.337473 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bd303-8509-4a49-b268-83f1e7769b70 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:28.408456 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:28.410431 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:28.411230 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:28.411688 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:28.412135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:28.413769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:28.525348 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:28.526811 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6ca8d-7f90-4d3a-81f4-f833733c8cd0 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:28.545458 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 744/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:30.655442 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:30.655997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:30.655997 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:30.657897 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 744/1488] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:30.663194 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843002-b19a-4f39-9772-b9dc4848def7 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:30.663665 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843002-b19a-4f39-9772-b9dc4848def7 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:30.664234 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843002-b19a-4f39-9772-b9dc4848def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:30.729088 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-8b843002-b19a-4f39-9772-b9dc4848def7 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:30.730344 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-8b843002-b19a-4f39-9772-b9dc4848def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:30.783372 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:30.827623 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 745/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:30.869062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:30.870637 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:30.904805 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:30.972919 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9fa4f-2e85-4734-b0ac-2bdc536885ad None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:31.045181 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:31.045798 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:31.046115 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:31.046395 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:31.048206 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:31.127680 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:31.157544 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:31.159006 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62d70-608e-4b9c-92d6-f6f7acf719fc service neutron] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:31.176633 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 745/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:33.434339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:33.434875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:33.434875 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:33.436159 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 746/1491] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jul 27 06:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:40:33.441444 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5901cb-3125-467d-aee6-4daa7b67d447 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:33.441857 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5901cb-3125-467d-aee6-4daa7b67d447 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:33.442041 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5901cb-3125-467d-aee6-4daa7b67d447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:33.507069 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-1d5901cb-3125-467d-aee6-4daa7b67d447 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:40:33.508281 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-1d5901cb-3125-467d-aee6-4daa7b67d447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:33.605147 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 746/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jul 27 06:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:40:33.646012 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:40:33.647567 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:33.751769 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019593e7-f1cb-47f4-b7d9-c89a24659dd2 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:40:33.822555 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:40:33.823162 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:40:33.823439 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:40:33.823712 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:40:33.825305 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:40:33.834895 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:33.934037 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:40:33.935504 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddf533-23ec-4f41-b35e-83ccdc926412 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:40:33.953905 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 747/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:40:43.603359 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=82 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:40:43.951545 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=50 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:13.408959 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:13.409589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:13.409589 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:13.484425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:42:13.485628 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:13.578829 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 747/1494] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:42:13.587112 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:13.587684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:13.587684 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:13.662484 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.auth.core [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:42:13.663981 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:13.758350 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 748/1495] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:42:13.762975 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:13.764724 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:13.775903 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7db13-1b35-4975-a71a-6e6224e882a6 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:13.836001 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:13.870502 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:13.871120 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:13.871402 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:13.871677 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] PATH_INFO: `/v3/domains` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:13.872360 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 06:42:13.872662 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:13.873691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING py.warnings [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:13.873691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: warnings.warn(msg) Jul 27 06:42:13.873691 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: Jul 27 06:42:13.874611 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:13.886262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 748/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Wed Jul 27 06:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 06:42:13.895162 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:13.904193 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf2fab-80de-4f30-ac0f-a7f1e77717c8 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:13.965365 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:14.000262 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:14.000878 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] REQUEST_METHOD: `POST` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.001164 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.001444 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] PATH_INFO: `/v3/projects` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.002148 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:14.003886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING py.warnings [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:14.003886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: warnings.warn(msg) Jul 27 06:42:14.003886 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: Jul 27 06:42:14.005074 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:14.063946 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 749/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Wed Jul 27 06:42:13 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:42:14.068179 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:14.069753 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:14.103736 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc803b3-6df8-4b12-83a7-e9772c35ccbe admin admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:14.196461 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:14.197192 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.197478 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.197756 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] PATH_INFO: `/v3/users` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.198462 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:14.199526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING py.warnings [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:14.199526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: warnings.warn(msg) Jul 27 06:42:14.199526 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: Jul 27 06:42:14.200721 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:14.243971 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 749/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jul 27 06:42:14 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 06:42:14.248005 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:14.250609 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:14.313250 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121ef7-1628-4613-859b-4688ffdb026c admin admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:14.395673 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:14.408135 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:14.408757 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.409039 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.409312 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] PATH_INFO: `/v3/roles` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.410073 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:14.411137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING py.warnings [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:14.411137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: warnings.warn(msg) Jul 27 06:42:14.411137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: Jul 27 06:42:14.412021 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:14.425138 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 750/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 06:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:42:14.429221 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:14.430776 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:14.442230 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47db96-4f38-4a06-b719-9593191b716c admin admin] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:14.466571 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:14.536927 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:14.537470 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.537749 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.538054 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] PATH_INFO: `/v3/roles` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.538738 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:14.539751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING py.warnings [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:14.539751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: warnings.warn(msg) Jul 27 06:42:14.539751 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: Jul 27 06:42:14.540624 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df31a1-c621-4a4d-8018-5b5f8c5627db admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:14.552750 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 750/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Wed Jul 27 06:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 06:42:14.556978 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:14.558518 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:14.591420 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5a21-1e4e-4078-9b5a-045b9d361631 admin admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:14.692568 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:14.693247 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.693527 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.693804 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] PATH_INFO: `/v3/projects/0d1f38e6eaa44dcc9786725481577437/users/7dcf6269c6ee4812bbf33ef47d7af46f/roles/4372496541ff48858e12197c1d6b1e71` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.718797 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1f38e6eaa44dcc9786725481577437, user_id=7dcf6269c6ee4812bbf33ef47d7af46f, role_id=4372496541ff48858e12197c1d6b1e71)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:14.719865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING py.warnings [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:14.719865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: warnings.warn(msg) Jul 27 06:42:14.719865 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: Jul 27 06:42:14.721779 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9b9af-c79a-4b31-b7dd-26312e2f65d9 admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:14.746513 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 751/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Wed Jul 27 06:42:14 2022] PUT /identity/v3/projects/0d1f38e6eaa44dcc9786725481577437/users/7dcf6269c6ee4812bbf33ef47d7af46f/roles/4372496541ff48858e12197c1d6b1e71 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:42:14.758683 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970e35-addd-4197-91e5-92e83e37d069 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.759025 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970e35-addd-4197-91e5-92e83e37d069 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.759304 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970e35-addd-4197-91e5-92e83e37d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.817485 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-ed970e35-addd-4197-91e5-92e83e37d069 None None] MFA Rules not processed for user `7dcf6269c6ee4812bbf33ef47d7af46f`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:42:14.818695 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-ed970e35-addd-4197-91e5-92e83e37d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:14.894202 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 751/1502] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 06:42:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:42:14.907433 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.907743 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.908019 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] PATH_INFO: `/` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.909565 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 752/1503] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jul 27 06:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:42:14.915016 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5c22-333f-4c7b-93e2-0a07c1206fa0 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:14.915128 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5c22-333f-4c7b-93e2-0a07c1206fa0 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:14.915425 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5c22-333f-4c7b-93e2-0a07c1206fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:14.946361 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:14.989582 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-b0ec5c22-333f-4c7b-93e2-0a07c1206fa0 None None] MFA Rules not processed for user `b9f7fa9ae17a4ed7a717ed044eabca83`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:42:14.990824 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-b0ec5c22-333f-4c7b-93e2-0a07c1206fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:15.065137 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 752/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jul 27 06:42:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:42:15.072942 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:15.074417 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:15.124857 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:15.178473 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008f5ba-1df8-4902-85a6-2c489d1779d0 None None] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:15.252407 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9f7fa9ae17a4ed7a717ed044eabca83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:15.253028 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:15.253302 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:15.253572 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:15.255200 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:15.350893 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:15.352310 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:15.355713 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:15.370346 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 753/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:42:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:42:15.415110 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f534d-9019-4412-b61e-a0d9e58e7582 None None] REQUEST_METHOD: `GET` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:15.415401 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f534d-9019-4412-b61e-a0d9e58e7582 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:15.415707 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f534d-9019-4412-b61e-a0d9e58e7582 None None] PATH_INFO: `/` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:15.417038 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 753/1506] 10.222.0.36 () {52 vars in 830 bytes} [Wed Jul 27 06:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 06:42:15.420592 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:15.421833 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:15.517832 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e411b-7e7a-4e44-bb57-44ed9d21a966 service cinder] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:15.610060 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:15.610683 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:15.610902 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:15.611182 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] PATH_INFO: `/v3/users/7dcf6269c6ee4812bbf33ef47d7af46f` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:15.624242 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dcf6269c6ee4812bbf33ef47d7af46f)` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:15.625315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: WARNING py.warnings [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:15.625315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: warnings.warn(msg) Jul 27 06:42:15.625315 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: Jul 27 06:42:15.626551 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:15.636119 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=100 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:15.697767 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dcf6269c6ee4812bbf33ef47d7af46f'} {{(pid=86820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:15.698339 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dcf6269c6ee4812bbf33ef47d7af46f'} {{(pid=86820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:15.709686 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dcf6269c6ee4812bbf33ef47d7af46f'} {{(pid=86820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:15.728075 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dcf6269c6ee4812bbf33ef47d7af46f'} {{(pid=86820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:15.728528 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.notifications [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dcf6269c6ee4812bbf33ef47d7af46f'} {{(pid=86820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:15.746382 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 754/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Wed Jul 27 06:42:15 2022] DELETE /identity/v3/users/7dcf6269c6ee4812bbf33ef47d7af46f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:42:15.751431 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] REQUEST_METHOD: `POST` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:15.751913 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:15.752113 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:15.808436 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.auth.core [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] MFA Rules not processed for user `cc98442be8894ee2a2dc3bab2659ad72`. Rule list: `[]` (Enabled: `True`). {{(pid=86819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 06:42:15.809651 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:15.883377 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=100 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:15.896224 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 754/1508] 10.222.0.36 () {58 vars in 958 bytes} [Wed Jul 27 06:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 06:42:15.905689 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Authenticating user token {{(pid=86820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:15.906919 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:16.009335 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473e26-8885-4d9b-9858-4e277cce7e4c admin admin] Validating token access rules against request {{(pid=86820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:16.064785 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:16.084334 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb171d91fc464ac1ae2ade235f5c7608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f26e9454984c23b5d9935e5a1e7109', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:16.084948 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:16.085227 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:16.085500 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:16.087137 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.fernet_utils [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:16.194590 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:16.195932 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5052f4-be25-4e5f-af75-fbbae6485cb3 service neutron] RBAC: Authorization granted {{(pid=86820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:16.214566 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: [pid: 86820|app: 0|req: 755/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jul 27 06:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 06:42:16.550835 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] Authenticating user token {{(pid=86819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 06:42:16.552062 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.fernet_utils [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 06:42:16.563947 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74a8f9-16d7-4349-a439-cd0905a882b5 None None] Validating token access rules against request {{(pid=86819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 06:42:16.660471 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc98442be8894ee2a2dc3bab2659ad72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d277a3b91684c12b74997019b6d7be2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 06:42:16.661262 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 06:42:16.661631 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 06:42:16.661877 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] PATH_INFO: `/v3/projects/0d1f38e6eaa44dcc9786725481577437` {{(pid=86819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 06:42:16.673951 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1f38e6eaa44dcc9786725481577437)` {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 06:42:16.674991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: WARNING py.warnings [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 06:42:16.674991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: warnings.warn(msg) Jul 27 06:42:16.674991 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: Jul 27 06:42:16.676169 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] RBAC: Authorization granted {{(pid=86819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 06:42:16.734002 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1f38e6eaa44dcc9786725481577437', 'request_id': 'req-10d97a99-748d-47a9-a0d2-76706395d09b'} {{(pid=86819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:16.746772 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1f38e6eaa44dcc9786725481577437', 'request_id': 'req-10d97a99-748d-47a9-a0d2-76706395d09b'} {{(pid=86819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:16.753282 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1f38e6eaa44dcc9786725481577437', 'request_id': 'req-10d97a99-748d-47a9-a0d2-76706395d09b'} {{(pid=86819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:16.780479 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] The token cache is being invalidate because project 0d1f38e6eaa44dcc9786725481577437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 06:42:16.780771 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:16.781293 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG keystone.notifications [None req-10d97a99-748d-47a9-a0d2-76706395d09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 06:42:16.782692 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: [pid: 86819|app: 0|req: 755/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Wed Jul 27 06:42:16 2022] DELETE /identity/v3/projects/0d1f38e6eaa44dcc9786725481577437 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 06:42:26.212221 dv-cinder-850989-4-iscsi devstack@keystone.service[86820]: DEBUG dbcounter [-] [86820] Writing DB stats keystone:SELECT=62 {{(pid=86820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 06:42:26.778083 dv-cinder-850989-4-iscsi devstack@keystone.service[86819]: DEBUG dbcounter [-] [86819] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}